Log all bash sessions for all users

      No Comments on Log all bash sessions for all users

Saving for myself:

To log all bash sessions for all users in Linux, put this at the end of /etc/bash.bashrc:

if [ ! -d "/some/log/path/$USER/" ]; then
  mkdir -p /some/log/path/$USER/
fi
if [ -d "/some/log/path/$USER/" ]; then
  [[ "$(ps -ocommand= -p $PPID | awk '{print $1}')" = 'script' ]] || { script -f /some/log/path/$USER/$(date +"%Y-%m-%d_%H-%M-%S.%3N")_shell.log && exit ;}
fi

Thanks to the following two posts for the details to get the behavior I wanted (that behavior being: to log the sessions, and then exit the parent shell that spawned the script command as expected when I exit the script command, instead of simply exiting the script command and leaving me in the original bash shell):

Running remote graphical Linux programs through Windows Subsystem for Linux

I have recently been using the Windows Subsystem for Linux on my Windows 10 laptop at work, with Ubuntu running as the distribution in the subsystem.

I recently had a need to use a graphical (X11) program running on a remote Linux VM. I was able to successfully run the program remotely, with the display on my local laptop by doing the following:

  • Install the Xming X11 server on Windows
  • Open a WSL Ubuntu shell (bash) and run export DISPLAY=localhost:0
  • run ssh -X username@remotehost.fqdn
  • on the remote host, run xprogram

I’ll say, the display of the program popped up *immediately* and was very performant throughout my use of it. I was quite impressed. (I have done this same kind of thing before with Xming + Putty + X11Forwarding, and the performance was never so bad to be unusable, but it was never all that impressive, either. This felt like I was running the program locally! Granted, the remote VM was running on hardware running in the same building in which I am working, but the performance was impressive nonetheless compared to a variety of other solutions I have used in the past, including “native” XWindows on a local Linux system with SSH X11Forwarding.)

In order to make this available to me in future sessions of my local WSL Ubuntu, I added export DISPLAY=localhost:0 to my local ~/.bashrc

This allows you to run other X programs from within the local WSL as well! For instance, I installed some basic programs in my WSL Ubuntu instance with sudo apt install x11-apps and then I could run xeyesand get the output on my monitors.

Win!

vCenter Server 6.7 Update 2 announced

vCenter Server 6.7 Update 2 has been announced, with several interesting things, but some highlights are:

  • Web UI interface for the Converge Tool used to collapse external Platform Service Controllers back into an embedded mode. (Only available on the command-line prior to this update.) – that will make it more straightforward to convert those external PSCs into an embedded deployment (that supports embedded linked-mode.)
  • File-Based Backup and Restore will now have NFS & SMB options! This is great – you can setup a backup schedule for the vCenter server and have it automatically save a backup file to a standard Windows file server. Much easier for many people than requiring an FTP, SCP, or HTTPS destination.
  • Also: New to the vSphere Client is Developer Center with Code Capture and API Explorer. Of most interest is the Code Capture option:

Senior Technical Marketing Architect, Kyle Ruddy says it all in a recent post; “If you’ve ever wanted to see how tasks performed in the UI would look as code, this is it.

Enjoy!

VMware Export VM as OVA or OVF

      No Comments on VMware Export VM as OVA or OVF

This post is mainly so I have a quick reference for myself.

To export a VM as a folder of files (OVF) or a single OVA file, you can use the Export-VApp method in PowerCLI. (It works on single VM’s, too, not just VApps. Just make sure to use the “-VM” parameter.)

From the documentation:

Specifies a destination path to the file system location where you want to export the vApp or the virtual machine. If the value of the Destination parameter is a folder, the vApp or the virtual machine is exported to a container folder (OVF). If the destination is a file, the vApp or the virtual machine is exported in OVA format.

https://vdc-repo.vmware.com/vmwb-repository/dcr-public/557aee4d-26f9-4810-bc12-b1cf22401c8f/ca9f89cf-cdea-4e9c-9e50-f214cf902c1b/doc/Export-VApp.html

But despite that behavior, I like to be precise with my intent when issuing commands, so I also use the -Format parameter that indicates if you expect to export an OVF or OVA.

So, to export a single VM as a folder of files with a referential OVF file, use the following format:

Export-VApp -Destination 'C:\some_path\VM_NAME_export\' -VM VM_NAME -Format Ovf

And to export the VM as a single OVA file:

Export-VApp -Destination 'C:\some_path\VM_NAME.ova' -VM VM_NAME -Format Ova

Most of this was gleaned from the PowerCLI reference documentation at https://code.vmware.com/web/tool/vmware-powercli

Change SciTE’s default open filter from “All Source” to “All Files”

Posting this mainly to keep it for myself, but making it public to benefit anyone else looking for it as well.

By default, SciTE‘s open dialog box is set to “All Source,” which means to show a list of all files it considers “source files” (files that contain program source code.)

I wanted my copy of SciTE to display “All Files” in the open dialog box by default, and then I could select “All Source” if I wanted that more narrow selection.

To do this, I opened SciTE, navigated to the menus Options -> Open User Options File, which opened ~/.SciTEUser.properties and I added this one line:

top.filters=$(all.files)All Source|$(source.files)|

This specifies that it should use the $(all.files) filter by default, then show the “All Source” option, then all the individual $(source.files) options (Ada, Bash, C++, etc.)

FYI, $(all.files) is defined as the following, depending on platform (I use a Linux distribution, so my platform is PLAT_GTK):

# Each platform has a different idea of the most important filters
if PLAT_WIN
	all.files=All Files (*.*)|*.*|
	top.filters=All Source|$(source.files)|$(all.files)
if PLAT_GTK
	all.files=All Files (*)|*|Hidden Files (.*)|.*|
	top.filters=All Source|$(source.files)|$(all.files)
# As OS X only greys out filtered files, show all as default
if PLAT_MAC
	all.files=All Files (*.*)|*.*|
	top.filters=$(all.files)All Source|$(source.files)|

Peace. Love. Hope. Strength. Solidarity. Persistence.

Peace, Hope, Love. Do not despair. (Or rather, feel free to work through any disappointment and despair, but then progress to more productive thoughts.) Let us be the leaders we need, those that can have compassion, can share with others, can fight for our rights and freedoms and defend the rights and freedoms of others. Let us be positive and willing to live in this world, talking with our family, friends, and neighbors; discussing divisive issues in a civil manner, and trying to see where the other person is coming from regarding a divisive issue, and then also standing beside each other in strength and solidarity where we agree.

It is difficult, to be sure – there are many issues that have two (or more) “sides” to them where no side is completely in the wrong. How do we work through these things and come to a conclusion aside from “agree to disagree”? I say we need to keep talking. Try to understand why each person has the view they do, and see where you can agree with some or all of their points while pointing out your own concerns, and how those concerns run counter to the other view, even though you see and respect that view, too. Perhaps we can come to agreement on how to move forward in a way that is for the benefit of all, and incorporates the ideas of both those “sides.”

Above all, please do not immediately jump into an “Us” vs. “Them” mindset. Even if someone is screaming their opposing view in your face – they came to that view through their own experiences and knowledge in this world. Perhaps you can learn something from them. Perhaps you can eventually teach them something about how you came to your view. But if you immediately jump to a “Oh s/he is one of THEM,” then nothing will ever be accomplished except division and negativity.

Let us all try to come together.

Two of my friends expressed sentiments regarding Donald Trump winning the presidential election that I’d like to share (I’ve gotten their permission):

From Jessi Mae:

I can’t help but feel angry at people who spent this campaign talking about how they were going to hold their noses and vote this year. We needed hope and optimism, like we had with Obama. What we got was Donald Trump as president.

But it is done now, and we can’t change the past. I’m going to let myself be angry and upset, but only for a little while. We have things to do.

Please, let’s just love each other a little bit more. Let’s show the bullies of the world what we can do when we lift each other up, rather than tearing each other down. Let’s lie a little less. Let’s not be afraid to be a little vulnerable.

Now, our jobs are more important than ever. We must be the change we NEED to see in this world. Our children are watching.

From Liam O’Shea:

Well, I didn’t vote for him, but we’ve managed to chose Trump as the next President. Personally, while I don’t think there’s a nuclear holocaust in the offing, i do feel we’re likely to regress in some key areas (science, education, environment, civil rights), but there may still be room for some good to occur, and hey, i could always be way wrong (but i don’t think i am).

This result really shouldn’t be a surprise. During the democratic primary, one of the big talking points was that there was no likely scenario in which Clinton beats Trump. The view was that only Sanders could do it. Democrats chose Hilary, and that narrative suddenly changed to Trump has no path to 270. That new narrative persisted until last week, when all of a sudden, there’s a very narrow gap in the polling numbers.

I still don’t see any coherent policy from President-Elect Trump, and I honestly don’t believe he understands how government actually works. That could mean the first two years of his presidency are mired in learning how things are done and sounding out policy. That might actually work to our benefit as a nation – if only one thing can get done at a time, then judging fallout from new policy should be a little easier, which could make Presidential policy course corrections easier (or more easily highlight the need for changes at Mid-Term)

Depending on who you talk to, this election is either a ringing endorsement of Democracy (when everyone gets out to vote, the corrupt political establishment can be overcome), or it can paint democracy in a very damning light (Look what happens when everyone is given an equal say in how everything is done – a lunatic gets elected by appealing to the fears and divisive nature of society), and I honestly think there’s truth in both sides of that argument.

Going back to my statement from yesterday, if you aren’t content with the results of yesterday’s election, then begin today the process of identifying your upcoming incumbents. Learn the platforms. If you come across negative press (or really positive press even) citing a sound bite or quoting only a small portion of a talk, go looking for the source. Read the speech transcript, read their opinion papers, etc. Get to know what is going on around you and look for opportunities to change.

In the meantime, your countrymen-and-women are not your enemies. They aren’t sub human, or scum, or slime, for holding different opinions or having differing priorities when it comes to education, healthcare, or social policy. It also isn’t ok to lord over someone that some other candidate was elected. Be Adult about this stuff.

Tone down the “I’m leaving” rhetoric. Emigration only helps lock in whatever path we’re now on.

For better or worse, this is now the world we live in. Democrats, pick your battles. Republicans, be true to the citizens that elected you and those that didn’t, and remember that our founders were agents of compromise, if nothing else.

Fix VolumeUp|VolumeDown|Mute/Unmute keyboard keys in Lubuntu

Mostly for my own reference; to get the Volume Up/Volume Down/{Mute|Unmute} keys to work on my Microsoft Natural Ergonomic Keyboard 4000 in Lubuntu (Ubuntu with LXDE/Openbox), I had to modify the lubuntu-rc.xml file to indicate the correct commands to execute when those keys were pressed.

Initially, I thought that the system wasn’t identifying the keys correctly, so I was using the xev tool to see what keycode was being sent to the system, but I found that the system already understood the keys properly. (They were identified as XF86AudioRaiseVolume, XF86AudioLowerVolume, and XF86AudioMute.)

The problem was that I am using the pulse audio daemon and the incorrect commands were being executed to adjust volume/mute/etc.

So, I edited ~/.config/openbox/lubuntu-rc.xml, looking for “volume” and commenting out the old command, substituting in the correct command as follows:

    <!-- Keybinding for Volume management -->
    <keybind key="XF86AudioRaiseVolume">
      <action name="Execute">
        <!-- <command>amixer -q sset Master 3%+ unmute</command> -->
        <command>amixer -D pulse set Master 3%+ unmute</command>
      </action>
    </keybind>
    <keybind key="XF86AudioLowerVolume">
      <action name="Execute">
        <!-- <command>amixer -q sset Master 3%- unmute</command> -->
        <command>amixer -D pulse set Master 3%- unmute</command>
      </action>
    </keybind>
    <keybind key="XF86AudioMute">
      <action name="Execute">
        <!-- <command>amixer -q sset Master toggle</command> -->
        <command>amixer -D pulse set Master toggle</command>
      </action>
    </keybind>

Then I ran the command openbox --reconfigure to reload the configuration file.

Success!

OpenSSH internal-sftp chroot logging on Ubuntu

I’m putting this here mainly to store it for myself, so these are going to be rough notes until I have time or inclination to review and edit this post.

Note: this solution was formulated on a server running Ubuntu Server 14.04 LTS

To setup sftp chroot directories for sftp-only users, and yet still log all actions & transfers like a “xferlog”, you need to somehow get the chrooted user to be allowed to write log data into the main syslog daemon.

First of all, you need the following in your sshd_config, and make certain to pay attention to the “ForceCommand” line, as I was missing that at first, and therefore, you are not changing the logging for the chroot users, just the non-chroot users:

Subsystem sftp internal-sftp -l VERBOSE -f LOCAL6

# Next section restricts users in sftpusers group to chrootDirectory

Match Group sftpusers
  ChrootDirectory %h
  ForceCommand internal-sftp -u 002 -l VERBOSE -f LOCAL6
  AllowTcpForwarding no
  PermitTunnel no
  X11Forwarding no

Now, you want to have a line like this in your /etc/rsyslog.d/50-default.conf or other /etc/rsyslog.d/*.conf file, in order to direct the sftp verbose logs to a specific file:

local6.*			/var/log/sftp.log

Then, you have a couple of options on how to proceed. The first way I got working was to add another listening socket in the rsyslogd config, with this line:

$AddUnixListenSocket /home/username/dev/log

You could also use this method:

mkdir /home/username/dev
touch /home/username/dev/log
chmod 511 /home/username/dev
chattr +i /home/username/dev
mount --bind /dev/log /home/username/dev/log

Either one should work for you – the question is how you want to handle the situation, especially if you have either a large number of users, or a constantly changing user base; you’ll have to orchestrate either adding the special mount point and then having a lot of extra mounts, or adding a lot of extra listening socket files in rsyslogd. I leave it to you to decide which is better for your situation.

I also saw a method that didn’t work for me, but was for HP-UX, and might work for Solaris, too, over here: http://community.hpe.com/t5/Security/ssh-subsystem-internal-sftp-chroot-and-logging/td-p/5272802


Updated Thursday 31 August 2017 by David J. Pryke:

Following up on this, if you want certain information that is only available at the higher-verbosity log levels, such as INFO or VERBOSE, but those levels are too verbose, you can pipe the syslog output to a separate command that can filter certain things out of the stream that are very “chatty”, or filter the data stream down to only the exact info you want.

First, edit the syslog configuration line to pipe the output to a separate command:

local6.*			|exec /usr/local/sbin/sftp-server-log-filter

Then create a script to filter in/out any data you want.

For example, to filter out lines that have the words “opendir” or “closedir” on them (since this can be voluminous in sftp-server’s logs on VERBOSE):

while read foo
do
    echo $foo|/usr/bin/egrep -v "opendir|closedir" >> /var/log/sftp.log
done

Or to filter the output down to ONLY a few lines that are interesting with you, such as those that begin with “open “, “close “, and “session “, try:

while read foo
do
    echo $foo|/usr/bin/egrep ".*\]: open |.*\]: close |.*\]: session " >> /var/log/sftp.log
done

Note that the regex patterns with “.*\]: ” are because these lines typically look like:

Aug 31 11:02:09 hostname sftp-server[53433]: session opened for local user username from [192.168.1.2]
Aug 31 11:02:24 hostname sftp-server[53433]: open "/usr/home/username/filename" flags READ mode 0666
Aug 31 11:02:24 hostname sftp-server[53433]: close "/usr/home/username/filename" bytes read 1147 written 0
Aug 31 11:02:27 hostname sftp-server[53433]: open "/usr/home/username/filename" flags WRITE,CREATE,TRUNCATE mode 0644
Aug 31 11:02:27 hostname sftp-server[53433]: close "/usr/home/username/filename" bytes read 0 written 1147
Aug 31 11:02:28 hostname sftp-server[53433]: session closed for local user username from [192.168.1.2]

Import Xen VM from .xva file via cmd-line or GUI

Importing a Xen VM from a .xva file

Import/Restore a Xen VM from a backed-up image file, likely a backup taken of a VM prior to the destruction of said VM for any reason.

Import via command line (faster and preferred method)

This method took 2:33 (2 minutes 33 seconds) to import a VM that had an 8GB HDD, of which 6 GB was used

  1. Ensure you have mounted the location with the .xva file. For instance, if it is a remote NFS store, you may want to check the current mount points, or execute:

    mount -t nfs 10.1.1.250:/mnt/backups/xenbackups /mnt/backup
    
  2. Take note of the filename & path that you will use for restoration/import, such as:

    /mnt/backup/Test-VM-2016-02-17_12-23-01.xva
    
  3. If you don’t have shared storage in your Xen Pool, or if you wish to place the VM on a specific server or Storage Repository (SR), execute the command:

    xe sr-list
    

    which will produce output such as this:

                uuid ( RO): 73a394e5-2a60-a13d-9c42-4987d81c1a77
          name-label ( RW): DVD drives
    name-description ( RW): Physical DVD drives
                host ( RO): XEN-SRV-4
                type ( RO): udev
        content-type ( RO): iso
    
                uuid ( RO): 3b3346b2-d08a-90a8-bc13-321e0fe988d9
          name-label ( RW): Removable storage
    name-description ( RW): 
                host ( RO): XEN-SRV-4
                type ( RO): udev
        content-type ( RO): disk
    
                uuid ( RO): 69659292-ca18-3875-8c3b-1c9873db1dc0
          name-label ( RW): Local storage
    name-description ( RW): 
                host ( RO): XEN-SRV-4
                type ( RO): lvm
        content-type ( RO): user
    

    Take note of the uuid of the “Local Storage” of the host on which you wish to place the VM. In this case, if we want to place the VM on “XEN-SRV-4,” then we will need the uuid “69659292-ca18-3875-8c3b-1c9873db1dc0

  4. Using the uuid & filename from above, you can then execute the vm-import command thus, preferably on the physical host where you are importing the VM, to minimize network traffic and potential slowdowns caused by passing packets from one server to another unnecessarily:

    xe vm-import filename=/mnt/backup/Test-VM-2016-02-17_12-23-01.xva sr-uuid=69659292-ca18-3875-8c3b-1c9873db1dc0
    

    If you want or need to retain the VIF MAC addresses (such as for a VM that gets its IP Address via DHCP), ensure that you specify the preserve=true option such as:

    xe vm-import filename=/mnt/backup/Test-VM-2016-02-17_12-23-01.xva sr-uuid=69659292-ca18-3875-8c3b-1c9873db1dc0 preserve=true
    
  5. If you wish to have the VM start automatically when the Hypervisor/Host boots, ensure you re-enable the autostart option, as it does not get restored when importing a VM from a file. For that, you’ll need the uuid of the VM (xe vm-list can help you there) and then you need to execute:

    xe vm-param-set uuid=<UUID-OF-VM> other-config:auto_poweron=true
    

Import via Windows XenCenter GUI (slower method)

This method took 22:40 (22 minutes 40 seconds) to import a VM that had an 8GB HDD, of which 6 GB was used, an order of magnitude slower than the command-line method above

  1. First, map a drive to the location with the backups (For instance, map X: to \\backups.my.domain\xenbackups) which you can optionally do via the GUI, or with command line:

    net use x: \\backups.my.domain\xenbackups /user:<USERNAME>
    
  2. Take note of the filename & path that you will use for restoration/import, such as:

    X:\Test-VM-2016-02-17_12-23-01.xva
    
  3. Right-click on the physical host you wish to place the VM onto (such as XEN-SRV-4), then select Import…

  4. Click Browse… to select the file containing the image you wish you import (X:\Test-VM-2016-02-17_12-23-01.xva) then click Next

  5. Select the Home Server for the VM (XEN-SRV-4 in this case), then click Next

  6. Select the storage for the VM HDD(s), which if you don’t have shared storage, will be the Local Storage of the Home Server, then click Import >

  7. Select final networking settings and finalize the import. The VM should start automatically after the import is complete.

  8. If you wish to have the VM start automatically when the Hypervisor/Host boots, you’ll need to ensure you re-enable the autostart option, as it does not get restored when importing a VM from a file. For that, you’ll need the to login to the shell (SSH/terminal) of one of the Xen Hosts, and will need the uuid of the VM (xe vm-list can help you there) and then from the shell, you need to execute:

    xe vm-param-set uuid=<UUID-OF-VM> other-config:auto_poweron=true