Miscellaneous FAQ

Donald Kenney (donaldkenney@gmail.com)
Last Update: Sat Nov 5 16:13:00 2022

Introduction

This is a sort of ad hoc list of stuff that has taken me an inordinate amount of time to figure out because it is either :

Table of Contents

If anyone needs it, there is a more traditional Table of Contents at the end of this document
TKINTER (from Python)
Listbox cursor location Listbox Multiple Selections Empty Listbox cursor location Overriding Tkinter Attributes
Tkinter Clipboard Handling Creating a New Tkinter Widget A Simple Scrolled Listbox Widget for Python Reconstructing a Menu
Launching Python subtasks Printing using Tkinter
PYTHON
Lambda Evaluation in Callbacks Invoking a function from both a button and a bound keystroke Switching Between TKINTER Toplevel Entities Organizing a large graphical interface
Deleting objects from a sequence How to insert "" into empty fields in CSV Square root of a number Launching Zombie Tasks
Pointers Functions Copying a List Importing Your Own Modules
Convert list of lists to dict Determine what statement called a function Free Variables Python-Calling Unix Dialog
[Unix one liners #ONELINERS
THUNAR
GUI File Copy
XFCE
Desktop Security Terminal invocation Autologin
BASH
bash configuration Environment Variables Saving stderr to a Log File cd to directory
Stuffing keystrokes into programs Embedding code into a script (heredoc)
KONQUEROR/FIREFOX
Listing Plugins Turning Access Keys Off
KONSOLE
Setting konsole to use line drawing characters
HTML
How to link to a Google Book
UNIX
Files ordered by size Full text search of directories Manually unpack .tgz and .tar.gz files df reports 100% Disk Usage
How to update a Unix Computer Time from via NTP How to pause until user presses ENTER key SLACKWARE Kernel Compilation Enable Netbios Name Resolution
Using arecord to capture sound card output Downloading from a git repository
Hardware Problems
Mouse will move only one way on diagonals
Household Problems
Black Mold Removal
Other
Accessing Michael Crichton's Essays


TKINTER (from Python)

Listbox click events-cursor location

Problem: If you bind the Listbox Click event <Button-1> to a callback and use cursorselection() in the callback, you get the previous cursorselection because the callback is invoked before the cursor position is updated.

Solution: Bind to the <<ListboxSelect>> event instead.


Listbox click events-multiple selections

Problem: Clicking a task in one Listbox clears the selections in other listboxes.

Solution: To prevent this behavior, add exportselection=0 to the ListBox declaration. (No, I have no idea, what you lose by doing this.)


Listbox click events-empty lists

Problem: How to determine whether a listbox is empty or has no selection. The size attribute for an empty listbox is set to 1, not zero. int(curselection()[0] throws an Index Error exception if nothing is selected.

Solution: The listbox curselection() property will return an empty list [] if the listbox is empty or no entry is selected. Documentation that says an empty string will be returned appears to be wrong.


Overriding Tkinter Attributes

Problem: The normal method of overriding existing attributes is to define a new class with the same name as the overridden attribute. It's not easy to see how to do this with Tkinter -- especially if Tkinter has been accessed by the often recommended from Tkinter import *.

  1. Subproblem 1 If the attribute to be overridden is a Widget, it is far from obvious what the call should look like.

  2. Subproblem 2 If the new attribute needs to reference the original attribute there is no way to do that because there is no longer a name in the namespace bound to that attribute.

Solution:

  1. Solution to Subproblem 1: To find the existing call, find Tkinter.py file you are importing in a library somewhere and search it for the attribute you are overriding. If it is a Widget method, it will probably inherit its call from the Widget object or the BaseWidget object and it's call will look like Widgetname (self, master, ** kw).
    1. self is, of course, an identification parameter unique to classes that is not explicitly set in the calling program.
    2. master is the Tkinter parent object -- typically a Frame or Toplevel container object.
    3. kw is a dictionary of keyword, value pairs. The double asterisk apparently tells Python to convert the keyword list in the call to a single dict object.

      Note that some Tkinter calls include a cnf parameter that seems to be an artifact of pre Python-1.3 when Tkinter parameters were passed as a dict explicitly created by the caller. I don't have a clue how that works.

  2. Solution to Subproblem 2: If one is coding from scratch, use:
      import Tkinter                                    #import a graphics interface
      from   Tkconstants import *                 #Make the graphics constants local
    

    instead of from Tkinter import *.

    If one wishes to invoke widgets without prefixing a Tkinter object, one can define a list of unprefixed names bound to prefixed names.

      import Tkinter                                    #import a graphics interface
      from   Tkconstants import *                 #Make the graphics constants local
      from   Tkinter import (Button, Entry, Frame, Label, Listbox, Menu, Radiobutton, 
                 Scrollbar, StringVar, Tk)                  #Make widget names local
    

    If from Tkinter import * has already been used, just follow from Tkinter import * with import Tkinter This allows attributes to be accessed either with or without a prefix. If there are differing versions of the attribute, the new one will be the one without a prefix. The original one will be the one with the prefix.

    The original attribute is now accessible as Tkinter.attribute Here's a simple override of the Button widget assuming that Tkinter.Button has somehow been made visible.

      class Button(Button) :
          def __ init __(self, master=None, cnf={}, **kw) :
              Tkinter.Button.__ init __(self,master, kw)
    

    If you are wondering if one can just from Tkinter import *, then set button_link = Button and use, button_link instead of Tkinter.Button, the answer is yes, but button_link probably has to be linked (bound) outside the Button class and declared as global in the class.


Tkinter Clipboard Handling

Problem: The default Clipboard bindings for a Tkinter Listbox work erratically. Shift-Ins works, Ctrl-Ins, Ctrl-Del, ^C and ^X work, but don't leave the copied/cut text selected even though it is in the Clipboard. ^V doesn't work at all. Apparently, the some of the keys are bound incorrectly and/or to objects that do not generate key stroke events.

Solution: Bind the Clipboard keys to the event_generate methods of the textbox and specify <<Copy>>,<<Paste>>,or<<Cut>> events as appropriate. For example, if the Listbox name is t:

t.bind("<Control-Insert>",lambda x: local_event_generate("<<Copy>>"))

HOWEVER this may lead to other problems under Linux:

Problem: If clipboard key settings are overridden as suggested above one may find that the PASTE keys (e.g. shift-insert) work fine with text CUT or COPIED within Python-Tkinter but paste text twice when used with text CUT or COPIED from external programs. In addition, if there is marked text at the cursor, it is not overwritten as is normally the case for paste operations. And finally, cut operations disappear the first character after the text marked for cutting.

Solution: The double paste problem seems to be due to the internal paste logic being called twice referring to different clipboards -- presumably due to a previous binding on the keystrokes. The solution seems to be to return 'break' after generating the first PASTE event thus suppressing processing additional keystroke bindings. The marked text issue can be fixed by generating a CUT event before pasting. The extra character deletion can be fixed with a local_cut event handler similar to that used for PASTE. I suspect that these problems occur only on Unix platforms.

  def local_paste(object) : 
    temp = object.clipboard_get()                 #save the clipboard temporarily
    local_cut(object)                                     #get rid of marked text
    object.clipboard_clear(); object.clipboard_append(temp)
    object.event_generate("<<Paste>>"); return 'break'
  
  def local_cut(object) :
    object.event_generate("<<Cut>>"); return "break"
  
  def rebind_clipboard(a):
    """Rebind CUA clipboard control keys to a specific object (Python's
       generic bindings don't always work properly)."""
    a.bind("<Control-Insert>",lambda x: a.event_generate("<<Copy>>"))
    a.bind("<Shift-Insert>",lambda x: local_paste(a))
    a.bind("<Shift-Delete>",lambda x: local_cut(a))
    a.bind("<Control-c>",lambda x: a.event_generate("<<Copy>>"))
    etc


Creating a New Tkinter Widget

Problem: Attempting to create new tkinter widget classes doesn't work right. Gets error messages at run time about a missing tk attribute.

Solution: The problem seems to be in the way Python and Tkinter are interfaced. Widget methods are acted upon by a method named tk that is apparently a Tcl interpreter. Just specifying inheritance does not suffice? (why not?) New widgets must explicitly initialize whatever existing widget class they inherit their properties from. They do that by calling the Widget's __ init __ method in order to get a tk attribute.

This leads to a second problem which is that multiple inheritance doesn't seem to work for tkinter widgets in Python. It is possible to work around that by inheriting as many attributes and methods impossible from one base class, then accessing any additional attributes and methods via their associated widgets within the object. If, for example, we create a widget x that has a listbox called l within a frame f and the widget inherits its properties from type Frame (and remembers to call Frame._ _init_ _()), we can access pack() as either x.pack() or x.f.pack(). But we can only access listbox insert as x.l.insert(...whatever...) where ...whatever... is the parameter list. Alternatively, one could explicitly declare the methods that need to be provided, then simply call the appropriate component method internally. e.g. def insert(self,...whatever...): self.x.l.insert(...whatever...)

Upon further thought, I think maybe we can do multiple inheritance with Tkinter widgets, but we probably need to explicitly call the _ _init_ _ entry to each of the base classes -- probably in reverse order of precedence. Perhaps I'll try it when I get time.

See https://mail.python.org/pipermail/tkinter-discuss/2008-August/001605.html


A Simple Scrolled Listbox Widget for Python

Problem: Tacking a vertical scrollbar onto a Tkinter Listbox widget is possible, but it's hard to remember how.

Solution: Here is a ScrolledList class that can be instantiated as a ScrolledList object. Accessing Listbox attributes and methods of the object requires a couple of extra characters in the attribute/method name, but otherwise using the class should be straightforward.

  class ScrolledList(Frame) :
    """ Defines a Listbox with an associated Scrollbar.  It seems not to
     be possible to use multiple inheritance with Tk widgets.  This 
     widget acts like a Frame.  grid(),pack() and other Frame methods
     can be used directly.  Listbox and Scrollbar methods must be 
     accessed indirectly via their names in this class -- l and s.  That
     is to say that if x is defined as x = ScrolledList(... that x.pack()
     is a legitimate operation, but that operations on the embedded 
     Listbox or Scrollbar must take the form x.l.insert(... or x.s.whatever
  
     invocation is obj_name = ScrolledList(Tkobject, height=n, width=n) 
     where Tkobject is the Tk object container where the scrolled list 
     box will appear and height and width are the dimensions of the 
     Listbox. """
    #NOTE: **from tkinter import * ** should have been run before this 
    #class is encountered.  As I understand it
    #  .Imports are normally done at the front of Python scripts
    #  ."from x import .." is not desirable, but Tkinter is an exception
    #  .A second import statement does no significant harm
    #NOTE: A peculiarity of the txt2tags markup that I use requires a
    #  couple of extra spaces in things like **__ init __**.  You'll need
    #  to remove the spaces manually in order to use the code
  
    def __ init __ (self, Tkobject, height=4, width=50) :
      """This code is executed whenever a new scrolled list object is
         created (instantiated) """
      Frame.__ init __(self)                #Make sure we are a widget
      self.f = Frame(Tkobject)                    #We Create a frame
      self.s = Scrollbar(self.f,orient=VERTICAL)
      self.l = Listbox(self.f, height=height, width=width, 
                       yscrollcommand=self.s.set, exportselection=0)
      self.s.config(command=self.l.yview) #Hook scrollbar to listbox
      self.s.pack(side=RIGHT, fill=Y); self.l.pack()     #Good to go
      # ---- Adding some methods -----------------------------------------------
      def bind (self,event,action) :                         #define bind method
          self.l.bind (event,action)
      def curselection(self) :                       #define curselection method
          self.l.curselection()
      def focus_set() :                                        #focus_set method
          self.l.focus_set()
      def insert (self,where,what) :                    #define an insert method
          self.l.insert (where,what)
      def pack (self) :                                    #define a pack method
          self.f.pack()
      def see (self,which) :                                #define a see method
          self.l.see(which)
      def selection_set (self,which) :            #define a selection set method
          self.l.selection_set (which)


Reconstructing a Menu

Problem: There is no provision that I can find in Tkinter to determine what menu items have been created or in what order. For dynamically created menus, this means that a separate persistent state array may need to be built and maintained if one needs to keep track of what the menu being displayed looks like.

Solution: The following code fragment will create a list called menu that shows the menu labels and separators titles currently being displayed for a menu called M_MENU:

      menu = []; M_MENU_len = M_MENU.index('last')
      for i in range(M_FILE_len+1) : 
          try : menu.append(M_FILE.entrycget(i,'label'))
          except TclError : menu.append('Separator')


A (Unix) printer Interface using Tkinter

When I wanted to add a printer interface to my Personal Information Manager (PIM) script, I couldn't find an example. I came up with the following which works in Slackware 12.1/Python 2.5.2/Tkinter Revision 50704. There is some stuff that could be done more cleanly. And I doubt it will run under Windows without considerable work because it depends on system interfaces that behave differently on different OSes.

  Note 1:  notify() is a function that displays messages to the user.  A
  different script would probably use a different error handler.  I find notify()
  to be useful.  I've included the code for notify() after the printer code
  
  Note 2:  The routine can print either from a text area in the application
  or from the clipboard.  Obviously, you don't have to do things that way 
  if you don't want to.
  
  # ==== TK_Printer CLASS ======================================================
  class TK_printer() :
      """A TKinter Widget that provides a printer interface for Linux"""
      """Attend to imports: (by convention, these are done in the main program
         right at the top.
      import os                                       #Needed for os.popen calls
      from Tkinter import *
        or something like:
          import Tkinter                            #import a graphics interface
          from Tkconstants import *           #Make the graphics constants local
          from   Tkinter import (Button, Entry, Frame, Label, Listbox, Menu,
             Radiobutton, Scrollbar, StringVar, Tk, TclError, Toplevel, IntVar)
                                                        #Make widget names local
             import tkMessageBox                         #Get Do This-OK? Dialog
             import tkFileDialog                     #Get file open/save dialogs
      """
      
  
      def show(self) :                     #This is the actual printer interface
          def local_popen(cmd,mode) : 
              """MAGIC: give the SIGCHLD signal to popen, to avoid close error
              but take it back after to avoid filling Linux computers with zombie tasks
              -- None of that is MY fault"""
              import signal, os, subprocess as sub
              signal.signal(signal.SIGCHLD, signal.SIG_DFL)
              #probably should use shlex.split but I'm a bit irked by this 
              #subprocess thing (the examples didn't work) and split should
              #be good enough for this app.  Or, don't need split if shell=True
              #specified.
              pp = sub.Popen(cmd.split(),stdout=sub.PIPE,stderr=sub.PIPE)
              output,errors = pp.communicate()
              signal.signal(signal.SIGCHLD, signal.SIG_IGN)
              if output or errors :
                  return((output + os.linesep + errors).rstrip(os.linesep).splitlines())
              else : return('')
              """
              #=================================================================
              #for those who prefer something deprecated, compatible with older|
              #  Python versions, and simpler, try this:                       |
              # -------------------------------------------------------------- |
              # | import signal,os                                             |
              # | signal.signal(signal.SIGCHLD, signal.SIG_DFL)                |
              # | result = os.popen(cmd,mode).readlines()                      |
              # | signal.signal(signal.SIGCHLD, signal.SIG_IGN)                |
              # | return (result)                                              |
              #-----------------------------------------------------------------
              # But annoyingly, this sequence somehow breaks the dialog module 
              # menu option.  Need to look into that ... someday
              """
          
          self.T = Toplevel()           #T is the Tkinter screen for the widgets
          Label(self.T, text="PRINTER SELECT", underline=8).grid(row=0,column=1)
          # ==== populate printer list via lpstat command ======================
          printers = local_popen('lpstat -a','r')        #get a list of printers
          for i in printers :
              if 'Unable to connect to server' in i :
                  notify("Can't find printers.  cupsd not running?")
              if i : break
          else : notify("No printers found")
          pl = Listbox(self.T,height=len(printers), width=50)    #Define Listbox
          for ix,i in enumerate(printers) : pl.insert(ix, i.split(' ')[0])
              #Note that we just discarded all the details on printer availability without even
              #looking at it.  (was in i[1] ... etc)
          pl.grid(row=10,column=1)                          #Display the Listbox
          pl.focus_set()
          
          # ==== Handle the Radio Buttons for Text Source ======================
          text_source = IntVar()    #IntVar is a TKinter entity that is bound to
                     #Radio Buttons and will be updated by clicks on the buttons
          text_source.set(1)                           #Set Text area as default
          Radiobutton(self.T, text="Print from teXt Area", variable=text_source, value=1,
          state=ACTIVE,underline=13).grid(row=25, column=1) #TxtArea Btn
          Radiobutton(self.T, text="Print from cLipboard", variable=text_source, value=2,
          state=ACTIVE,underline=12).grid(row=26, column=1) #Clipbrd Btn
          
          # ==== Callbacks invoked by Tkinter Events ===========================
          # ---- Cancel the Print Request --------------------------------------
          def cancel_callback (KeyCode=None) :  self.T.destroy()
          # ---- Perform the Print Request -------------------------------------
          def print_callback (KeyCode=None) :
  #            if debug: print 'Print Routine Called'
              # ---- legality checks before printing ---------------------------
              nogo = 7 ;   #sum of flags 1=Ptr no select,2=No source,4=empty src
              out_txt = ''
              while nogo != 0 :
                  if pl.curselection() == () : notify ("You must select a printer")
                  else : nogo = nogo & 6       #Printer select OK-try text source
                  if not text_source.get() :  notify ("No Text Source Selected")
                  else : nogo = nogo & 5                    #Text source selected
                  if text_source.get() == 1 : out_txt = p.text
                  if text_source.get() == 2 : out_txt = t.clipboard_get()
                  if not out_txt : notify ("No Text to Output")
                  else : nogo = nogo & 3
              # ---- At this point, we are good to go --------------------------
              # ---- Get a temporary file name and save the printout -----------
              temp_file = local_popen('mktemp','r')[0].strip()  #Get unique name
              F = open(temp_file,'w')                       #write the temp file
              for i in out_txt : F.write(i)
              F.close()
              # ---- Print the temp file and discard it ------------------------
              printer = pl.get(pl.curselection()[0])
              # ---- Dispatch the print job ------------------------------------
              x = local_popen('lpr -P ' + printer + ' -T text -r ' +  temp_file, 'r')
              notify ("Print job submitted")
              cancel_callback()                    #Exit more or less gracefully
          # ---- Define the rest of the GUI stuff for printing -----------------
          Button(self.T, text='Cancel', command=cancel_callback,underline=0).grid(row=30,column=1)
          [self.T.bind('<Key-'+i+'>', cancel_callback) for i in 'cC']
  
          Button(self.T, text='Print', command=print_callback,underline=0).grid(row=30,column=2)
          [self.T.bind('<Key-'+i+'>', cancel_callback) for i in 'pP']
          
          [self.T.bind(i, lambda x: text_source.set(2)) for i in 'lL']
          [self.T.bind(i, lambda x: text_source.set(1)) for i in 'xX']
          [self.T.bind(i, lambda x: pl.focus_set()) for i in 'sS']
  # For windows lpr=PRINT,lpstat=PRNMGR -- no equivalent to mktemp?
  

  # ==== notify FUNCTION =======================================================
  def notify (msg, secs=3) :
      """notify (msg, [secs=n]) -- displays a notification box for a specified
         number of seconds (default=3 seconds)"""
      import time                              #need the time module to do sleep
      nf = Toplevel(K)                              #Define the notification box
      nf.protocol("WM_DELETE_WINDOW", nf.destroy) #Catch close by window manager
      Label(nf,text=msg,width=len(msg)).grid(row=80,column=1)  #Disp the message
      nl1 = Label(nf,text=str(secs))                   #Display a second counter
      try :
          for i in range(secs,-1,-1) :                     #Count down n seconds
              time.sleep(1)                                       #Wait a second
              nl1.config(text=str(i))                 #Update the second counter
              nl1.grid(row=81,column=1)                         #Display the box
              nf.update()                                    #Update the display
              nf.tkraise()                            #Bring it to the forefront
      except TclError :     #Assume any TclError comes from window manager close
          pass
      nf.destroy()                          #Finished, kill the notification box
      K.update()                                              #Update the screen


Lambda Evaluation in Callbacks

Problem: In Python Lambda operations are evaluated at run time -- which is OK most likely. And Callbacks with parameters require Lambda -- which is, I think, a weakness in the language. There are then three "times", compile time, run time when the callback is created, and a "callback time" when the lambda expression in the callback is actually evaluated. One result is that attempts to create an array of objects invoked via Callbacks will create Callbacks that (at best) all reference the final item in the array.

In my case, attempting to create a variable length array of menu items that simulated mouse clicks on an Array of Radio Buttons looked like this

    for i in range(len(XF)) :
      mf.add_command(label="Tab "+str(i)+' '+XF[i],
          command=lambda: XT[i].event_generate("<Button-1>"), underline=4))

Runs OK, but all the menu items created act on the last entry in the XF array. It's actually a bit worse than that. They apparently all use the most recent value bound to i in the relevant namespace. Even if i was reused for something else entirely after being used to create widgets.

Solution: To get subscripts to reflect values they have to be decoded in advance then reflected by applying eval (or exec?) to the resulting expression.

    for i in range(len(XF)) :
      eval('mf.add_command(label="Tab '+str(i)+' '+XF[i]+'",
          command=lambda: XT['+str(i)+'].event_generate("<Button-1>"),
          underline=4)')


Invoking a function from both a button and a bound keystroke

Problem: Menu options and such in Tkinter generally call functions using a command=... option in the Menu, Button, Radiobutton, etc definition. Typically no parameters are passed to the invoked function which is convenient because passing parameters requires the use of lambda functions. Lambda functions cause most people's heads to hurt and have odd limitations. The same function can easily be bound to a keystroke event. Unfortunately, Python-Tkinter automatically passes a keystroke code to the called function when it is invoked by a bound keystroke. So now we need a called function which has no parameters when invoked from a Mousebutton press, but one parameter when invoked with a keystroke. An exception will be thrown if the number of parameters doesn't match the function definition.

Solution: Unlike many computer languages Python supports optional positional parameters.

    def useful_function ( dummy = 0 ) :
        ...
  
    Radiobutton(FrameName, test="Useful",..., command=useful_function )
    ...
  
    FrameName.bind("A",useful_function)

Will now run without throwing an exception whether the "Useful" menu item is clicked (no parameter passed) or the letter A is typed (one parameter passed).


Switching Between TKINTER Toplevel Entities

Problem: Occasionally, one may desire to completely hide one "screen" and replace it with another. It's far from obvious how to do that.

Solution: The following will do that.

Initial setup -- displays A

    ... specify toplevel entity A
    ... A.protocol("WM_DELETE_WINDOW", A_quit) #Catch close by window manager
    ... display it (A.grid() or A.pack()
    ... specify toplevel entity B
    ... B.protocol("WM_DELETE_WINDOW", B_quit) #Catch close by window manager 
    ... display it (B.grid() or B.pack()
    ... B.wm_withdraw()
  
    def B_quit() :
      B.wm_withdraw()
      A.wm_deiconify()

Switch to B

    A.wm_withdraw()
    B.wm_deiconify()

Switch back to A

    B.wm_withdraw()
    A.wm_deiconify()

Organizing a large graphical interface

Problem: I had a fairly large project with several dozen user controls. the Tkinter code to implement it evolved into an intractable shambles. Google searching for organizational ideas didn't turn anything up.

Solution:Here's what I came up with when I finally rewrote it.

Step 1 : Define a helper function that will create a button and/or corresponding menu item from a single call. Here's a simple version:

  # ==== Action Function =======================================================
  def Action (section,txt,cmd,y,x,menu,ul) :
      """Configure button and/or a menu item.  Return list of objects created"""
      o = [False,False]
      if menu :   
          o[1] = menu.add_command(label=txt,command=cmd,underline=ul)
      if section: 
          o[0] = Button(section,text=txt,command=cmd,width=12)
          o[0].grid(row=y,column=x)
      return (o)

What this does is pretty much cut the number of code lines for defining Buttons and Menu items in half. Two drawbacks:

  1. The Button and Menu Text must be the identical
  2. The Actions must be defined in the order they will appear in the menu

This particular implementation assumes that grid layout will be used. That's likely because the grid and pack managers are incompatible, and projects small enough to use pack without going insane shouldn't need this sort of organization.


Step 2: Define the container objects -- in my case Frames and Menus

  # Graphics containers --------------------------------------------------------
  ITEMS   = Frame(K,bd='2',relief=RIDGE)
  TEXT    = Frame(K,bd='2',relief=RIDGE)
  ...
  TOOLS   = Frame(K,bd='2',relief=RIDGE)
  
  # Menu containers -------------------------------------------------------------
  M_MAIN = Menu(K)
  M_FILE   = Menu(M_MAIN, tearoff=0)
  M_EDIT   = Menu(M_MAIN, tearoff=0)

Step 3: Define the Buttons and Menu items using the Action function. If necessary, explicit GUI definitions can be intermixed.

  Action         (TOOLS,"Open",  lambda: file_open(None),     12,90,M_FILE,0)
  b_save = Action(TOOLS,"Save",  lambda: db_save(p.file),     10,90,M_FILE,0) [0]
  Action         (TOOLS,"Save As", lambda: file_save_as(None),  14,90,M_FILE,4)
  ...
  M_FILE.add_separator() # - - - - - - - - - - - - - - -  Separator line in menu
  Action(TOOLS,"Exit/Quit",   lambda: quit(None),          17,90,M_FILE,1)

Many actions will never be referenced in Python code and don't need to be bound to names. But in this case we have a Save button whose background color will be changed when we think a Save will eventually be required. We need to bind it to a name so we can change its background color attribute later.

Step 4: Follow the GUI controls (Action calls, explicit definitions of Text boxes, Listboxes, Entries, etc) with any additional key/event bindings that Tkinter doesn't set up on its own.

  # ---- FILE Key Bindings -----------------------------------------------------
  for i in ("<Q>","<q>","<x>","<X>","<Alt-F4>") :
      M_FILE.bind(i, quit)
  K.bind("<Alt-F4>", quit)
  ...

Step 5: Code any functions required to support the actions and bindings. These are the objects of the lambdas, command=, etc elements. I found it easiest to code these in line during development then move them prior to the GUI stuff once the code was checked out.

Step 6: Display the containers defined in Step 1 and populated in Steps 2 through 5.

  # ==== DISPLAY THE CONTAINERS ================================================
  ITEMS.grid    (row=10,column=1,columnspan=4,sticky=W)
  TEXT.grid     (row=40,column=1,rowspan=5,columnspan=4,sticky=W)
  ...


Running Subtasks from TKinter Menu

Problem: If one launches a script that runs Python from a Menu in another Python program the event loop in the launching script will freeze until the launched script terminates. That can be prevented by appending an ampersand (&) to the launch command which detaches the subprocess as a separate task. But although the launched subprocess is able to output to the launching task console, it can't input from that console. Any raw_input statements will report EOF when reading a line.

Solution: if one wants to be able to input from the console to both the original process and the subprocess, one can launch the subprocess in a separate Linux console using one of the many terminal emulations (rxvt,xterm,Terminal, etc). e.g.

os.system('konsole -e bash -c "python -i my_python_script.py"')

Why konsole -e bash ...? I found that some scripts (I forget which) didn't run properly unless I specified which shell to use.


PYTHON

Deleting objects from a sequence

Problem: If you try to loop through a string or list in Python and delete specific elements based on some criteria, you will very likely find (not necessarily immediately) that elements after deleted elements are not checked for deletion and/or that an index out of range error occurs. That's because the iterator for the loop is created before the iteration starts. There is no way for it to know that if the (n)th element is deleted, the new (n)th element must be tested next. It simply moves on the the (n+1)th element. Simply reversing the list or string does not solve the problem, it just changes which elements are skipped over.

Solution: There are several solutions that will work for loops that potentially delete elements from the sequence.

1. One possibility is to iterate backwards so that any deleted elements are outside the region still to be examined.

1.1 Simply reversing the iterator may work: ** for i in reversed(range(len(example))) :** ...

1.2 Another way of creating a backwards iterator for i in range(n-1,-1,-1)

1.3 And another for i in a[::-1]

2. Another solution is to simply create a new list without the deleted elements. List (or generator?) Comprehensions can do this. e.g. new_example_list = [ i for i in old_example_list if not ... ]


How to insert "" into empty fields in CSV

Problem: Although Python's CSV module will read a CSV (Comma Separated Variable) file that has empty fields that look like , "" , it won't write them. Converting a CSV module created file back requires reading the file back and inserting , "" , strings in place of ,, strings. But there are three pitfalls:

It is probably possible to create a tricky expression that appends extra fields to the front of the comma separated line, does the conversions, then removes any initial or terminal detritus. A simpler (and possibly faster) solution is simply to forget the CSV module and write desired line in the desired format.


Square root of a number

Problem Unlike many computer languages, Python doesn't have a square root operator built into the language. That doesn't mean that you can't do square roots. To get the square root of x:


Launching Zombie Tasks

Problem If a python script launches tasks via spawn,etc. in Unix and sets P_NOWAIT in order to avoid blocking, the launched task will become a zombie when it terminates. That is to say that it will remain in the process tables and will be reported as "defunct". If the launching task is used a lot, it can create dozens or hundreds of these zombies. These processes will not go away until the Python script is terminated.

Solution This occurs because Python does not automatically handle the signal generated when a child process terminates. The Unix kernel is waiting for Python to acknowledge that it knows the child process has died. Adding

  import signal
  signal.signal(signal.SIGCHLD, signal.SIG_IGN)

near the start of the launcher script will allow the kernel to kill the terminating child task normally. If the script also uses P_WAIT, it will be necessary to bracket the calls where P_WAIT is used with signal.signal(signal.SIGCHLD, signal.SIG_DFL) and signal.signal(signal.SIGCHLD, signal.SIG_IGN) calls to prevent the signal when the wait is completed from being ignored.


Python Data Handling

Problem: Python memory management is "different". Python works with objects, not data in the conventional sense. Even assignment operations that look the same as assignment in languages like FORTRAN or C really aren't. Technically, what looks like assignment (e.g. a = 2+2 )is actually binding of a "variable name" ( a ) to an object (that contains the integer value 4). And yes, now that you ask, a single object -- the integer 4 for example -- can be, and very likely is, bound to many names. This leads to some problems for some people sometimes.

Pointers

It is said that Python does not support pointers and doesn't need them. That's largely true. Probably the most common use of pointers in languages that use them is to access sequential data. For the most part using Python's slice notation on sequence objects (strings,lists,etc) does the analogous thing in a possibly less mind bending fashion. Slice notation largely eliminates any confusion about the values of "variables" as opposed to the values of pointers to the variables.

For dealing with data tables or files laid out by C,FORTRAN,BASIC,etc programmers, there appear to be two not especially appealing alternatives -- treat them as sequences accessed via slicing, or use real pointers via the CTYPES package or something similar. For memory mapped IO or shared data in memory, I think CTYPES or something similar would be the only choice. Accessing mixed data by slicing is klunky. WARNING: CTYPES has a steep learning curve.

Emulating a Pointer(Aliasing)

One use of pointers in other languages is to minimize the use of complex expressions that have readability problems. For example, a pointer labeled text might be used in place of an expression like data_array[data_array_current_element][data_array_current_index]. There are ways to do this in Python.

Aliasing a Constant

A constant is something that will never be set, only used. These can be aliased in Python by binding a "constant name" to a lambda expression e.g. text = lambda : (data_array[data_array_current_element][data_array_current_index]). text can then be referenced normally e.g. print text

Aliasing a Variable

It is also possible to alias a variable that can be set (bound actually). It just isn't as easy. To do so, a class is needed.

  class Object_Pointer() :
    """This class is used to assign simple names to database variables"""
    #See http://www2.lib.uchicago.edu/keith/courses/python/class/5/
    def __ getattr __(self, name):
      global data_array, data_array_current_element, data_array_current_index]
      if   name == "db"      :
        return (data_array[data_array_current_element][data_array_current_index])
      elif name == ...   etc, etc, etc
  
    def __ setattr __(self, name, value):
      """Override the normal assign logic"""
      global XD,XX,X,x,XF 
      if   name == "db"      :
        (data_array[data_array_current_element][data_array_current_index]) = value
      elif name == "db_cur"  : ...  etc, etc, etc
  # ---- p ---------------------------------------------------------------------
  p = Object_Pointer()                                #Now, create a pointer set
  
  #At this point p.text looks like a normal Python variable.

NOTE: If del p.text might be needed, a __ delattr __ function will need to be defined. See http://www2.lib.uchicago.edu/keith/courses/python/class/5/

NOTE: One might think that the variables would need to be defined as variables in the class. Indeed, exactly the reverse. They need not to be defined in the class.

NOTE: In a real implementation, one would probably want to print a error message or throw an exception if a variable name is encountered that is not recognized.

Functions

Python binding can bind expressions or functions as well as values. That can be confusing. a = b[3]+1 binds a to whatever value is in b[3] with 1 added to it, not to the expression b[3]+1. e.g. b[3]=2; a=b[3]+1 ;b[3]=3.1416; print a** prints 3, not 3.1416 or 4.1416. But a can also be bound to a function b[3]=2; a=lambda:b[3]+1 ;b[3]=3.1416; print a tells where the lambda function is (note that a acts like a pointer here) whereas b[3]=2; a=lambda:b[3]+1 ;b[3]=3.1416; print a() prints 4.1416. As can be seen, if one really misses the complexity of C pointers, there are ways to approximate it in Python.

While binding a name to a function creates a pointer like entity that can be used in expressions, the inverse binding b[3]=2; a=lambda:b[3] ;a=6; print a,b binds a to 6, not b[3] to 6. And using a()=6 doesn't work either. So one can use lambda functions to alias some horrendous name to a simple term, but only for evaluation of expressions, not rebinding (resetting) except in the special case where all the horror is in an expression in a slice term. Didn't follow all that? That's OK. The bottom line is that Python programmers tend to define get and set methods for objects where other languages might use pointers to access and set the values.

There is a lot of discussion of assignment in sections 5.3 and 6.3 of the Python Reference Manual The document seems not to exist for Python after version 2.6 I believe that it still exists, but it's not clear where it has gone Most of the apparent complexity comes from the ability to assign lists of labels lists of values in a single operation and to augmented assignment operations such as +=

Solution: None -- that's the way Python works. Sometimes, it's the way you want it to work and it's a feature. Sometimes, it doesn't, and it's a nuisance.


Copying a List

Problem: Because of Python's use of binding instead of assignment, most people run into a problem sooner of later with:

  A = [0, 1, 2]
  B = A
  ...
  B.append(4)
  ...
  print B
  [0,1,2,4]   #as expected
  print A
  [0,1,2,4]   #Ooops

There's little doubt this is correct. But it is both unexpected and weird.

Solution: The solution often recommended is to import copy then bind B = copy.copy(A) see https://docs.python.org/2/library/copy.html. But B = list(A) will have the same effect and is simpler.


Importing Your Own Modules

A module can be pretty much any chunk of Python code. It can be imported and "compiled" with an import statement. Multiple import statements are harmless as an already loaded module will not be reloaded if a second import statement is encountered. There appear to be only two constraints:

If you wish to avoid having to put your local modules into libraries with "real" Python modules just to get it found, the following will load a module from the current directory:

  import os,sys
  sys.path.append(os.path.abspath(""))
  import your_module

But it might be preferable, to put your homemade modules in a directory somewhere then set the environment variable PYTHONPATH to point to that directory. "Importing" the desired code using execfile(filename) rather than import is another possibility (but execfile does not exist in Python 3)


Converting a list of lists to a dictionary

Problem: One of the more interesting capabilities of Python is the built in dict type which is content addressible. One use for a dict is for searching a list. Recently, I was compressing an intrasigent mess of code that did not seem to be working correctly into a short set of map and filter operations. That went fine until I needed to check if strings in one element were also in another list. The answer I came up with was to convert the other list to a dict and use name not in my_dict as the true/false variable in a filter command. Trouble is that I had to create an empty dict, then build a loop to build the dictionary. Nothing really wrong with that, but the asthetics of this loop in the midst of mapping and filtering left a lot to be desired.

Surely, there must be a way to convert a list of lists to a dict with one line of code.

Solution: dict is a function that will convert a two element list to a dictionary with the first element as the key and the second as the value.

  a = (1,2,3); b = (4,5,6); c = (7,8,9)
  given a list x = [a,b,c]
  my_dict = dict([i[0],i[1:] for i in x)
  
  will create a dict called my_dict with the first elements of a,b, and c as the keys.  e.g.  In this case my_dict will have three keys -- 1,4, and 7.  The values of key 4 will be a list (5,6)


Determine what statement called a function

Problem: Sometimes one wants to know what statement called a function without running a debugger. Debuggers can be difficult and obtuse when the program in question has a user interface of its own.

Solution: The following code, when inserted into a function will list the line number of the second element on the program stack -- the line number of the calling statement. This may not work in all Python implementations

      import sys
      print sys._getframe(1).f_lineno #Print line number of calling statement


Free Variables

Problem: Sooner or later one will probably encounter any one of a number of weird error messages involving "free variables". For example unqualified exec is not allowed in function 'xxx' it contains a nested function with free variables. These messages tend to be utterly baffling to the average Python programmer.

Explanation: Free Variables turns out to be Python-speak for variables that are not defined in either the global or local namespace. Free variables generally come about when module A calls *function* B which then calls *function* C -- where *function* means anything that has its own local namespace -- functions, classes, modules, exception blocks. Names ("variables") defined in A -- the GLOBAL namespace -- can be read pretty much anywhere. And they can be set anywhere if preceded by a GLOBAL declaration in the local namespace. Likewise names in C can be read and set in C, the LOCAL namespace, provided they are set before they are used. In general the free variables defined in B above can be read, but not set within C, but there are some other, often non-intuitive and rather mind-bending constraints. In general when Python throws an error involving free variables, it seems to be trying to tell us that it can't unambiguously determine what namespaces to look in for a name we are referencing and in what order to look.

Solution: Code will probably have to be altered either by refactoring or in the case of the exec command by specifying (a) namespace(s). Which code has to be changed? If you're lucky, it will be obvious.


Python-Calling Dialog

Problem: Unix has a nifty command line utility called dialog that can generate a wide variety of text mode widgets similar to gui widgets and nowhere near as annoying to work with. Sadly, invoking dialog from python is a bit complex. There is a a module called dialog.py that attempts to simplify using dialog from python. Unfortunately, dialog.py isn't as easy to use as one might hope. It doesn't abstract all dialog capabilities. And there seem to be several different versions of dialog.py with different subsets of dialog capability built in. I found debugging dialog.py calls to be quite frustrating. And there don't seem to be a lot of examples of dialog.py usage on the internet.

Solution: It is possible to call dialog directly using os.system (deprecated, but easy) or the subprocess module (more complicated, but encouraged by the Python folk). The advantages to doing so is that the dialog call can be debugged from a console, that all capabilities of your version of dialog are accessible, and that examples of how to use dialog are much easier to find than examples of how to use dialog.py.

For example, to output a message:

import os os.system('dialog --msgbox "Hello. I am a test message." 5 32')

or

import subprocess PROCESS = subprocess.Popen('dialog --msgbox "Hello. I am a test message." 5 32', shell=True) ; PROCESS.wait()

(NOTE: There are some examples of dialog.py usage at [http://pythondialog.sourceforge.net/])

One complication -- some dialog options return information such as menu selection information. The information is not output to stdout where one might expect it to be because ncurses which is used internally by dialog manipulates the screen by means of commands sent to stdout. Instead, dialog uses either the task return code or stderr or both to pass information back to the calling program. Return information in the return code from os.system can be accessed in a staightforward way.

ANSWER= os.system('dialog --yesno "Do you wish to be master of the universe?." 5 45')

Upon return, ANSWER will be 0 if Yes, 256 if No.

Using os.system getting information from stderr requires a temporary file. Append 2>tempfile to the end of the command, read it back, then delete the temporary file.

os.system('RESULT=dialog \-\-title "[ M E N U ]" --menu "Pick One ..." 15 50 4 Disaster "Disaster" Catastrophe "Catastrophe" Worse "Worse" 2>TEMPFILE') if RESULT : ... Need code here to deal with cancelled menu else : F = open("TEMPFILE","r") SELECTION=F.readlines() F.close() print SELECTION os.remove("TEMPFILE")

NOTE: the backslashes in the dialog command above are there to foil the document generator which would eat the -- if they weren't there. Probably they need to be deleted if you cut and paste the code.

subprocess will return a subprocess object instead of RESULT. You will need to extract the result code and menu selection, if any, out of that object if you choose to use subprocess.


Unix One-Liners in Python

Unix is generally run using a shell that allows simple tasks to be strung together working off the previous task's output. This scheme actually works mostly, but those "simple" tasks can be remarkably obtuse when they opt not to be remarkably helpful. It turns out to be quite difficult to do some simple sounding tasks using the standard Unix toolkit. Example, read a text file containing a list of files complete with paths, discard any lines where the file name ("basename") starts with a lower case letter. This is not anywhere near as easy to do as it sounds if you want the output lines to still include fully qualified paths along with the file name.

As it turns out Python can do the required filtering quite easily. The problem is how to insert a simple filter coded in Python into a Unix command stream. One possibility is to buffer the data through temporary files. But it'd be more convenient to use Unix pipes (indicated in Unix scripts by a |) that connect the output of one program to the input of the next. That turns out to be fairly simple also, if not exactly obvious.


THUNAR

GUI File Copy

Problem: You can't drag and drop a file within it's own folder to make a copy of the file when you drop it as you can in Windows Explorer and many other shells.

Solution: Copy the file to the Clipboard, right click on the folder and paste.


XFCE

Desktop Security

Problem: I am not much of a fan of the conventional security arrangement on Unix and Windows PCs. It is complex, difficult to administer, and provides little security if anyone knowledgable has physical access to the computer or, thanks to innumerable bugs in services and useful software, if the computer is attached to a network of any sort. It was intended to keep multiple users from trampling on each other's files and configuration settings on a shared computing resource. If managed by a very competent and careful administrator, it does that pretty well. There are many otherwise intelligent and thoughtful people, who believe that if the security system is followed rigorously, it will protect users from malware and data theft. I do not believe in that or in any other form of sympathetic magic.

Worse, well intentioned attempts to implement security using the permission/access control/password model actually makes the computers less accessible, less reliable and more error prone -- something that they do not remotely need. This is certainly true of Windows and Unix which I am familiar with. I assume it is true of Apple as well since the current Apple software is based on Unix.

Suppose that you won the lottery, learned to fly and bought a personal jet aircraft. Near your home in a major city, you keep the aircraft at a facility surrounded by fences and with access controlled by security people. When you go to fly, you must walk through a metal detector and your baggage is scanned and have your name checked against an access list. A small price to pay for securing your aircraft.

Your usual destination is your vacation compound in a remote corner of the Rockies. Only you use the landing strip there. Would you install a metal detector and baggage detector and hire security to people to hassle you whenever you flew from there? Of course you wouldn't. No one would. It'd be truly bizarre to do that.

The first situation is computer security in a typical business or government environoment. You need computer security if for no reason other than to protect users from each other. That's what Unix was designed to provide. The second situation is computer security on a personal computer. You'd have to be a bit of a masochist to use Unix security to protect such a machine. Nonetheless, many otherwise intelligent people think you should. Some damn well think you MUST. My position. They do not know what they are talking about and really would make both their and my lives easier if they stopped giving me and others bad advice.

My opinion:

Solution: So what's my better security model? I don't have one. But I am pretty sure that that no security model at all is at least as good as the model that is currently being touted. No security is about equally secure and substantially less aggravating.

If you really want a reasonably secure personal computer: . Keep the computer under lock and key . and use whole disk encryption with a password see http://tldp.org/HOWTO/html_single/Disk-Encryption-HOWTO/ . and do not ever connect the computer to a network -- any network.

If you do those things, the conventional security model is unnecessary. If you don't, it probably won't protect you.

If you must connect to a network, or cannot physically secure your computer, you can not, with current technology, have a secure system (for the most part communications based hackers break in by taking advantage of defects in the network interface software rather than by guessing your name and password). But, you may be able to improve your security by a few steps.


XFCE-TERMINAL

Terminal invocation

Problem: The man file for terminal provides virtually no information on the command line. The --help option does list the command line options, but the format shown for the -e (execute) option includes an extraneous equal sign and therefore doesn't work.

Solution: The command line for starting an instance of xfce4-terminal with a program running is:

xfce4-terminal -e "program param1 param2 param3"

Either single or double quote bracketing can be used for the program string.

on some (later?) versions something along the line of:

Terminal --execute program param1 param2 param3


AUTOLOGIN

User Autologin

Problem: How to start up processes automatically when XFCE Starts

Solution: Recommended: Make up a desktop file and put it in /etc/xdg/autostart/ or /etc/xfce/xdg/autostart/ (whichever your release wants). Use "Autostarted Applications" via the System Menu, or -- equivalently -- run /usr/bin/xfce4-autostart-editor from the command line and make sure that your desktop file is checked to be run.


Problem: How to Automatically log into a Linux system and start an X-Windows session. There are probably a hundred ways to accomplish an automatic logon into Linux. The problem is that there are probably ten thousand ways that won't work. The most common apparently is to use mingetty with it's autologin option. But not every Linux release includes autologin or will autologin with just mingetty added. The following works with Slackware and probably will work with any release using a bash shell.


BASH

bash configuration

modified 170117,180818

PROBLEM: For whatever reason, bash runs differently in a GUI terminal than in a console. For example, less when run in a console leaves the content of the last screen displayed. When run in a window, the content is erased.

ANALYSIS: You probably want to run different things when the bash shell is started as part of logging a user in than when it is run in a text console than when it is run in a terminal in the GUI. Linux accommodates that, but not very cleanly. And because it is difficult to see how the pieces fit together, Linux is likely to arrive with prompts and tasks like less behaving differently in a console than in the GUI. Different behavior may be necessary and desired. But it may also be unnecessary and annoying. It can be corrected. There are several scripts involved.

Solution: My recommendation: Assuming that you don't actually want non-login shells to have a different user interface than do login shells -

    # Use LS_OPTIONS with LS -- set defaults for less
    export LS_OPTIONS=' -F -b -T 0 --color=auto -H'
    alias ls='ls $LS_OPTIONS'  
  
    export MINICOM="-c on"          #Enable ANSI escape color control in minicom
    export MANPATH=/usr/local/man:/usr/man     #Set search path for man requests
    export LESSOPEN="|lesspipe.sh %s"                  #Define less preprocessor
    export LESS="X"             #disable termcap initialization when LESS is run


bash environment variables (a semi-rant)

Problem: Environment variables are a good idea, but implemented in a way that causes untold grief. They are named constants accessible to shell scripts. They are intended to configure a unix system for a specific task or set of tasks. Once set, they are available to the program/task that sets them and all its child tasks. For example PATH is an environment variable that contains an ordered list of places in which to look for an executable requested on a command line.

Unfortunately, the constraints on where environment variables will be visible (current task and child tasks) make setting and using them nightmarish. For example, in many Linuxes, there is a script in /etc/rc.d called rc.local that is used to localize a particular Linux configuration. So, you should set up your environment variables in rc.local? Nope, bad guess. You can set them, but when the rc.local script finishes, the settings will go away because stuff you will run later will not be children of rc.local. Not helpful.

Solution: Try not to use environment variables. They will, trust me on this, cause you aggravation sooner or later. But many subsystems -- Python for Example -- require them. And even with their limitations, they are sometimes necessary. Unless you are writing a complex of scripts/programs that will run as children of a master task, the place to create/tinker with environment variables appears to be /ect/profile which will be executed for every bash shell that is created.


Saving stderr to a Log File

Problem: By convention, Unix console programs output error messages to a separate destination (stderr) than normal output (stdout). Mostly this works fine, but when jobs run unattended, it is desirable to see the error messages. The easy choices are either to redirect stderr to a dedicated log file or to run the job in a console using a terminal that has a noclose (konsole) or hold (xterm) option. Neither choice is entirely satisfactory. If the job window isn't closed automatically, it must be closed manually which can apparently be easily done only with a mouse because there is no keyboard attached to the console. If stderr is redirected by appending >2 log_file_name to the command, error messages will be logged, but the file is restarted annew with every job that is run. That's not entirely satisfactory if multiple unattended jobs are being run as each would need its own log file.

It is not at all obvious how to append stderr output to a general error log file rather than creating separate log files from scratch.

Solution: It turns out the appending stderr output to a general error log is possible, but it's not all that obvious how to do it. One possible magic string to append to a command is 3>&1 1>&2 2>&3 | tee -a /log_file_name. There may be other ways to accomplish the same thing.

Translation:


cd to directory

Problem: It is possible in Unix shells to write shell or scripting language scripts to automate many tasks. But scripts that try to change to the working directory for some task then leave the user at a command prompt to run commands for development work do not work as one might expect. The problem isn't that one can't cd to a directory in a script. One can. The problem is that once the script completes, the working directory is changed back to the directory the script was launched from.

Solution: There are at least two:

Stuffing keystrokes into programs

Problem: Sometimes it is desirable to stuff keystrokes into a program that is being launched.

Solution: Unix has two tools for doing this expect for command line programs and xdotool for GUI programs. Here's an example of each.

expect

expect uses scripts and has the capability to monitor program outputs and insert responses to program queries. If, for example, your version of alpine is compiled without the option to specify passwords on the command line, you can still automate email login with a script that looks like this:

  #!/usr/bin/expect
  set timeout 30
  spawn alpine
  expect "PASSWORD:"
  send "your_password\r"
  interact

NOTE: if you, like me, have multiple mail accounts with different passwords courtesy of folks who attempt to improve my security practices, you can set up a separate user with their own exp file, then run alpine as them

  sudo -H -u second_user path_to_script/mail_second_user.exp


xdotool

xdotool is a command line tool that allows predefined keystrokes or mouse actions to be sent to processes running in X-windows. Here's an example of using xdotool to start konsole with three tabs open.

  preconditions:
   - set konsole to turn off display of hints when it it started (If you don't you'll have to insert keystrokes/mouse 
     actions to turn them off for the session)
   - set konsole to allow an alternate key combination besides Alt-Ctrl-N to create a new Linux Console.  (konsole ignores
     Alt-Ctrl-N from xdotool).  I used Shift-Alt-F8
   - install wmctrl -- a command line program that allows one to manipulate windows and workspaces("desktops")
  
  script:
  wmctrl -s 1                            #move to desktop 1
  konsole &            #start konsole in a detached process
  GEDPID=$!                              #get konsole's pid
  sleep 10                             #wait for it to open
  WINID=`xdotool search --pid $GEDPID | tail -1` #Get winid
  xdotool windowactivate --sync $WINID key --clearmodifiers
   --delay 100 shift+Alt+F8 shift+Alt+F8     #open two tabs
  

Embedding code into a script(heredoc)

Problem Many programs like terminal emulators (e.g. xterm,konsole) and scripting langages (e.g.Python) allow inline scripts to be incorporated on the command line invoking the program. In practice, this capability is severely limited due to the need to quote the inline script, deal with new lines, and indentation .

Solution Bash has a capability called a heredoc string that allows a block of formatted text to be inserted into a bash script using the << operator. The syntax is less than intuitive. Here's an example using python.

  python <<"ENDSCRIPT"
  import os,sys
  for i in range(3) :
      print 'hello world'
  ENDSCRIPT

NOTE that there are rules/conventions for handling variable expansion by bash and other contingencies.


KONQUEROR/FIREFOX

Plugins

Problem: Listing plugins in use is not a menu option

Solution: To list Plugins, Enter about:plugins in the "Location Bar" and press ENTER (Yes, now that you ask, this IS non-intuitive and probably not in accordance with anyone's ideas of a good user interface.).


Access Keys

Problem: Konqueror randomly displays numerous small boxes with letters in them superimposed on web pages.

Solution The boxes are part of the Access Key capability. Access Keys are a capability added to HTML in HTML4 -- the primary descriptive language used on the World Wide Web. Access Keys allow keyboard access to various areas of the web page. In Konqueror, they are supposed to be enabled by pressing the Ctrl key. Unfortunately, there is a bug thought to be in the linux kernel, that randomly sends Ctrl press events to konqueror when playing video or audio media. That causes the boxes to randomly turn on and off which most users find to be very distracting. Studies have found that the access keystrokes themselves conflict with established usages on almost all platforms/browsers. The capability has been deprecated in recent specification drafts. https://en.wikipedia.org/wiki/Access_keys

It is possible to turn Access Key usage off completely in Konqueror by adding

  [Access Keys]
  Enabled=false

to the .kde/share/config/khtmlrc file for the user who starts Konqueror, then restarting konqueror. This seems to work with Konqueror 3.9.5.


KONSOLE

Setting konsole to use line drawing characters

Problem: A number of text mode Unix programs, dialog for example, use line drawing characters in the user interface. Unfortunately, the konsole terminal emulator is likely to render these as characters like superscript 3s or umlated capital As instead of the desired line characters. That's ugly and confusing. Internet discussions of this -- at least the ones I found -- are remarkably unhelpful.

Solution: The line characters are only defined in certain fonts. In order to get them, one must go to Settings in the Konsole menu bar, select encoding, and select a font that includes line drawing characters (most don't seem to) Western European (IBM850) seems to be satisfactory. This will not redraw boxes already misdrawn but new boxes should be OK. One should then click Set As Default in the konsole Settings menu if one wants the change to be permanent.


HTML

How to link to a Google Book

Problem: When citing a reference to a Google Book, the resulting URL can become really unwieldy

Solution: Link by ISBN Number (usually available on the About the book page) Example: http://books.google.com/books?vid=ISBN0521349931

How can I link to specific Google Books


UNIX

Files ordered by size

Problem: How to get a list of files sorted by size

Solution: To get a list of the entities on disk sorted by size largest first:

du -B M / | sort -nr | less

(Takes a while) Or check this article for other options.

Full text search of directories

Problem: How to search large directories for files containing specific text

Solution:

for i in `find .`; do grep -H "text_to_search_for" $i; done

except that doesn't work right if you have files/directories with embedded blanks in the name (A practice that ought to be banned but probably can't be because Windows thinks spaces in file names are jim dandy wonderful so wine does also). It is extremely difficult to work around this in shell script.

An alternative that seems to work is to have find convert the blanks to nuls (ascii 0) then use xargs to convert them back then execute a command. At least I think that's what this does:

find . -print0 | xargs -0 grep "$1"

adding -i or --ignore-case to the grep command might make the search case insensitive


Manually unpack .tgz and .tar.gz files

Problem: Compressed file structures for Linux (e.g. most software releases) are typically packed up into a single file (a "tarball") with tar then compressed with gzip. I can never remember how to unpack them.

Solution: for file whatever.tgz

    gunzip -c whatever.tgz | tar xvf -
      or
    tar xvfz whatever.tgz

Unpacking can also be done as two operations:

    gunzip whatever.tgz 
    tar xvf whatever.tar

(Observe that gunzip eats the tgz file and replaces it with a -- generally much larger -- .tar file. Be aware also that gunzip is sometimes replaced with a different compressor/decompressor such as bzip/bunzip2)


df reports 100% Disk Usage

Problem: df and other disk space analyzers may report far more disk space in use than is actually used

Solution: This is sometimes due to trash folders created by modern, generally GUI, OSes. Trash is a repository for "deleted" files that can be recovered if the user decides that discarding the file was a mistake. Files from trash are discarded automatically when disk space is needed. There does not seem to be a simple tool that quickly reports disk usage exclusive of trash files although it is possible to work out usage with the df and du commands if one knows the name(s) of the trash directories. Other causes are run away log files (typically in /var/log/) and purportedly defective files placed in the /lost+found directory when file system checks (fsck) are run.


How to update a Unix Computer Time from via NTP

Problem: The permanent clock in PCs was designed to be inexpensive, not accurate. Many of them drift by seconds a day. After a few months, the clock can be off by 5 or 10 minutes. It is possible to reset the clock from a Network Time Protocol server on the Internet. The problem is that one can search for hours to determine the proper Unix command.

Solution: A command that will probably work is

ntpdate pool.ntp.org


How to pause until user presses ENTER key

Problem: Unix, unlike MSDOS and Windows, does not have a pause command that will cause a script to stop and wait for user action

Solution: The unix equivalent to a DOS pause command is:

read -p "message"

SLACKWARE Kernel Compilation

Problem: The Slackware Linux distribution has several different kernel configurations (smp, generic, huge). However, only one Source code configuration is available -- smp. Unfortunately, a number of applications including virtualizers like VMplayer, vbox, kqemu require compiling kernel modules, and those modules will not load (nor run if load is forced) if the source code is configured for the wrong kernel.

There is a readme file that explains how to tweak the source for use with a non-SMP kernel configuration, however the file is missing from many places where it possibly ought to be (the first six places I looked for it in fact).

Solution: I finally tracked a copy of the file at https://web.archive.org/web/20100715035116/http://slackware.osuosl.org/slackware_source/k/README.TXT. Just in case that goes away, I have placed a copy on this website Slackware-non-SMP-kernel-README.txt

The gist of the README seems to be:

I have tested this and it seems to work.


Enable Netbios Name Resolution

Problem: For reasons that elude me, Windows people seem to prefer GUI tools to command line tools that I personally find to be less confusing. That means that they want to run the (annoying) GUI remotely. Unix has a tool to do that -- rdesktop. However it appears from the internet that many people find that rdesktop won't connect to windows machines specified by Netbios name and will only work if the Windows computer's IP address is specified instead. Unfortunately IP addresses aren't always that easy to come by unless one has access to the Windows machine

Solution: It appears that in some cases, the problem is that windows name resolution needs to be turned on in unix. It in enabled by adding wins to the hosts: line in /etc/nsswitch.conf so that it reads hosts: files dns wins instead of just hosts: files dns. Once I did that rdesktop started recognizing netbios names.


Using arecord to capture sound card output

Problem: The command line sound card output capture program arecord doesn't work for most people most of the time.

Solution: The design of computer audio is an incredible mess which unix has managed to further exacerbate (no small feat). I have extensive notes on this that I may work up into an essay someday. For now: The canonical command to record the sound card output when using the ALSA sound subsystem is something along the line of:

  arecord -d 10 -c 2 -f S16_LE -r 44100 -t wav -D copy /tmp/foobar.wav

But alas, that won't work for most people, most of the time, because copy is an arbitrary name for an (imaginary) component in the sound system called a pcm. The pcm device has to be defined in a system wide configuration file -- /etc/asoundrc or a user specific version /HOME/user_name/.asoundrc which then apparently has to be referenced from a /etc/asound.conf or user specific /usr/share/alsa/alsa.conf file. No, I am not making this up. I wish I were.

What do you put into asoundrc? Purportedly

  pcm.copy {
   type plug
   slave {
   pcm hw
   }
   route_policy copy
   }

will work. (it didn't for me)

What do you put into alsa.conf? Damned if I know. Do you need to stop and restart the alsa sound system? Damned if I know.

An alternative is to simply specify the device id -- which is probably hw:0,0 unless your PC has multiple sound cards

  arecord -d 10 -c 2 -f S16_LE -r 44100 -t wav -D hw:0,0 /tmp/foobar.wav

which will very likely at least try to record something. But it will likely fail because the default sound configuration probably tries to record the microphone input, not the mixer input. To fix that

  amixer cset numid=27,iface=MIXER,name='Capture Source' 5

where both the numid and the final number are system dependent To get them, try:

  amixer contents | grep apture |

to get the numid, followed by amixer contents to list all the settings. I'd check the Capture Switch setting while I was there. If it is off, you probably will not record.

How do you fix the Capture Source permanently? My guess is that you edit /etc/asound.state

Finally, WAV format files are HUGE. To get something more rational, pipe the arecord output to a compressor such as oggenc (ogg.vorbis) or lame (mp3). But note that you may have to tell the compressor program about the bitrate, etc from arecord.

  arecord -d 10 -c 2 -f S16_LE -r 44100 -t wav -D hw:0,0 | oggenc  -o /tmp/foobar.ogg -

Isn't all that perfectly ghastly? (Audacious might work without all this ... or not).

[http://www.alsa-project.org/main/index.php/Asoundrc] [https://carthick.wordpress.com/2007/11/26/linux-recording-soundcard-output-using-arecord/]



GIT

Downloading from a git repository

Problem Git is a software configuration control tool developed by Linus Torvalds in 2005. It is widely used. It is no doubt a very good tool (I haven't used it to do a project). But configuration control is a broad field with a lot of complexity, so git has a lot of commands. Its help listing and man file are VERY difficult to follow. Increasingly, interesting open source software projects maintain their source at sites like github.com that do not seem to have a simple download button that will get you a copy of the code. So how do you download files from one of those sites? Search engines will lead you to a number of approaches whose common feature is that they don't seem to work if you (like me) don't know what you are doing and why.

Solution You need a copy of a git client. For unix, that's probably git. It is packaged with many linux distributions. If you are allergic to command lines, a search engine will turn up other git clients, many of them GUI and some for other OSes. You need the URI of the code you wish to download which will very likely have a scheme like https:// prefixed. To do the download, open a console (i.e run rxvt,konsole,xterm,Terminal,etc). At the prompt, type in the following command with the scheme on the git uri replaced with git://. destination_file is a file name on your computer

  git clone git://source_uri destination_file

It is apparently a common, but not universal, practice for git repositories to have subdirectories with compressed versions of the code tree -- zip for windows and/or tarball for unix.

Hardware Problems

Mouse will move only one way on diagonals

Problem: Erratic Mouse motion. e.g. it moves up and right normally, but down and right is iffy at best. Cursor motion in the affected direction is erratic, inconsistent, or it just won't move at all.

Solution: I've probably seen a thousand computer mice in the past two decades -- many of them misbehaving. But I encountered a problem recently that I've never seen before. The miserable thing would move normally up or right. But it was sticky and erratic moving down, or left. Cleaning would fix it for an hour or two, maybe a day, but the problem always came back. The mouses's internals were more or less conventional. The mouse ball rests on two shafts oriented at right angles. Each shaft has a slotted wheel that interrupts a "light" beam. The motion change is computed by counting the light interruptions. The ball is held against the shafts by a tiny tensioner consisting of a spring loaded plastic wheel that can move a bit to accomodate changes in ball height due to bumps in the mouse pad or minor changes in hand pressure.

The problem turned out to be that a small wad of dust, fabric fragments, etc was wedged under one side of the tensioning wheel. When the ball rotated in one direction, it pushed the wad away from the wheel, the wheel turned freely, and everything worked. When the ball rotated in the other direction, it pulled the wad under the wheel, the wheel jammed, and ball rotation became erratic. When I opened up the mouse, took the tensioner out, and washed it in isopropyl alcohol, the wad floated out. When the mouse was reassembled, it worked normally.


Household Problems

Black Mold Removal

Problem: Black Mold in the Bathroom

Solution: After noticing that we had an infestation of black mold around the bathtub. I decided to get rid of it. Soap didn't do much. Nor did various cleaning potions even though each and every one had the word 'magic' somewhere on the label. After checking the Internet, I tried vinegar and lemon juice (both weak acids) as well as baking soda (a weak base). It appears to me that black mold thrives on vinegar, lemon juice and baking soda. I passed on the internet's suggestion of a paste made from vinegar and baking soda. My 50 year unused chemistry tells me that will give me a paste of Sodium Acetate and a lot of CO2 bubbles. I don't think it will bother mold much.

What I finally used is bleach. Bleach applied every day for three or four months and allowed to sit for an hour or two at each application slowly disappeared the mold. The easiest way to apply it for me was with Clorox bleach pens that sell for a few dollars at grocery or hardware stores. Just make sure the bleach doesn't harm your wall finish or tub. Let it sit for a while, then just rinse it off with water. You can scrub if you wish. That will remove mold faster. But my experience was that scrubbing was more effective at removing grout and strengthening arm and wrist muscles than at mold removal.

Addendum Several years after I wrote the above, a bath and shower installer advised me that lightly spraying bleach on the bath/shower grout every two weeks or so would prevent mold from growing in the first place.


Other

Accessing Michael Crichton's Speeches

Problem: Michael Chrichton's Speeches are no longer accessible from his website. There is a link to his essays, but no link to his speeches. This is rather a shame because Crichton's speeches and essays are marvels of clear exposition even if one doesn't always agree with them.

Solution: The speeches are still accessible in the Internet Archives ("The Wayback Machine"). An index is available at http://web.archive.org/web/20060306061303/http://michaelcrichton.com/speeches/index.html. There are some later archives of the index.html file but they appear to be identical. It is possible that the material is still on the site under some other path/name, but can't (easily) be accessed because of the absent link. The links in the index.html file are provided via a byzantine structure of javascript that apparently was more interesting to the web site designer than the simple, reliable HTML alternative. They may not work in all browsers.

Tangential: Although I think Crichton's essays and speeches are -- on average -- well constructed and thought provoking, there a couple of claims that he makes that I believe are quite misleading:

. "DDT doesn't cause cancer" Probably true, but irrelevant. The problem is that the stuff is virtually indestructable. It was banned because it built up in the food chain and concentrated in the high level predators. It was causing birds such as Ospreys to lay thin shelled eggs the cracked before their offspring could develop. Why should you worry about that? You are a high level predator.

. "Secondary smoke doesn't cause cancer." Again probably more or less true, but leads to an incorrect conclusion. The concentration of carcinogens in second hand smoke is probably too low to cause many problems to those who breath it. The problem is that the particulates in the smoke include a broad spectrum of partially combusted hydrocarbons -- some of which possibly are carcinogens. The particles settle out on the furnishings where they build up over time into a surface film that might in fact be hazardous. This might not be a problem in immaculately clean surroundings like a hospital operating room, but in places like restaurants and bars ... Could be a problem with woodsmoke as well. It's not like anyone knows which component(s) in tobacco/tobacco smoke cause cancer.


Text Mode Table of Contents

For anyone who is using a browser that doesn't do tables or doesn't do them well, here is a traditional list of topics type table of contents


BOILERPLATE


Copyright 2006-2023 Donald Kenney (Donald.Kenney@GMail.com). Unless otherwise stated, permission is hereby granted to use any materials on these pages under the Creative Commons License V2.5.

This page will have been validated as Valid HTML 4.01 Transitional prior to posting on the web site. W3C Logo Image omitted.