Summer Nights

Let's make the world a better place using digital technologies!

Share Data using FTP Server

First create an appropriate user, with no shell access, belonging to ftp-users group and chrooted jail at /srv/ftp/xxxx:

useradd ftp-userxx -G ftp-users -s /bin/false -d /srv/ftp/xxxx

Now assign a password to the user, and make it the owner of the shared folder:
passwd ftp-userxx
chown -R ftp-userxx /srv/ftp/xxxx

Now copy the files to be shared into the respective folder. After that change the permissions to read-only. Be careful! we need +x in order to be able to do chroot:
chmod -R 544 /srv/ftp/xxxx

Assuming we are doing it using proftp, the following should be given:

  • proftpd running
  • Port 21 open
  • In /etc/proftpd/proftpd.conf file: RequireValidShell off
  • And to keep the users in the chrooted jail: DefaultRoot ~
Continue Reading

Reset USB 3.0 Device from the Linux Shell

If you are getting errors like:
cdc_acm 3-11:1.2: acm_submit_read_urb - usb_submit_urb failed: -12
or
xhci_hcd 0000:00:14.0: ERROR no room on ep ring
or
tty_port_close_start: tty->count = 1 port count = 0.

then most probably xhci (USB3) module is causing this problem. Generally you can fix this problem (at least temporarily) by removing and re-inserting the USB connector. However, it would be much better to do it from the command line. It is possible by doing:
echo -n "0000:00:14.0" | tee /sys/bus/pci/drivers/xhci_hcd/unbind
and then
echo -n "0000:00:14.0" | tee /sys/bus/pci/drivers/xhci_hcd/bind

Note that “0000:00:14.0” is the id of the malfunctioning USB device that can be different in your case. You can easily find this by inspecting your log (use dmesg).

Continue Reading

php does not show any output

If the php from the linux cli is not showing any output, then most probably it is crashing with a fatal error. It will not show any output if the errors are turned off in /etc/php5/cli/php.ini.

Turn on the errors by setting display_errors = On and you will find out what is causing the problem

Note: If php cli is not working because of a bad configuration, then pear will also not show any output.

Continue Reading

Using svn+ssh from Eclipse through Pageant & Plink under Windows 7 64-bit

So I just want to use my svn repositories that are already configured and ready to use with svn+ssh protocol. I can check them out through TortoiseSVN as I have explained in one of my previous posts. Now the next step is to integrate it directly in the Eclipse IDE. Here are the steps needed to achieve that:

  • Add new site http://subclipse.tigris.org/update_1.8.x through Help->Install New Software in Eclipse.
  • Install both Subclipse and SVNKit.
  • Create a new system variable SVN_SSH with the following value:
    C:\\Program Files\\TortoiseSVN\\bin\\TortoisePlink.exe -ssh -2 -A -l username.
    Take care to match the path and the username to your needs.
  • Restart Eclipse
  • Go to Window->Show View->Other and choose SVN Repositories.

  • Now go in the SVN Repositories tab on the lower side of the IDE and add the path to your repository (svn+ssh://yoursite/repo). If the authentication is working properly, you will now see you remote repository:
  • Now you can right-click on the desired project and Checkout to Eclipse to work on it. All the usual SVN commands will be available through Team menu.
Continue Reading

backur: A Configurable Python Utility for Optimized Incremental Backups

I have been on the lookout for an efficient small utility to incrementally backup my Linux server to an external drive. I have looked at various options like Back in Time and Backup PC. They are however too advanced and too automated for my purpose. My requirements are the following:

  1. Incremental backups
  2. Console based
  3. Preservation of file times, permissions, user and group
  4. Easy configuration
  5. Logging support
  6. Mirror mode

I have been using rsync for some time in bash scripts. As Python is becoming more and more my language of scripting, so I wanted to write it in Python. With that background information, here comes “backur”. backur is very easy to configure and use. backur.conf, as given below, is self-explanatory. It also provides customizable logging including log-file rotation. Th backup and source paths are very easy to enter, and allow spaces, thanks to shlex split method. You can run backur either manually, or through as a cron job, as it suits you.

#! /usr/bin/python

# Title:    backur.py
# Author:   Asif Iqbal (aiqbalrana@gmail.com)
# Date:     11OCT2012
# Info:     "backur" can be used to automatically backup given folder locations
#           to another backup location. backur is easily configurable through a
#           configuration file. backur uses rsync to achieve highly optimized
#           data transfers by doing only the incremental updates.    

import ConfigParser
import re, os
import glob
import logging
import logging.handlers
import subprocess
import datetime
import shlex

def splitNStripArgs(inputStr, charToStrip):
    data = inputStr.split(',')
    retData = []
    for item in data:
        retData.append((item.strip()).strip(charToStrip))
    return retData

# Read configuration data
confpath = os.path.dirname(__file__)
config = ConfigParser.ConfigParser()
config.read(confpath + "/backur.conf")

rsyncflags = splitNStripArgs((config.get("Config", "RSYNCFlags")),'"')
rsyncexe   = config.get("Config", "RSYNC")
folderlist = splitNStripArgs((config.get("Folders", "FolderList")),'"')
backuploc  = config.get("Config","BackupLocation")
backupsize = int(config.get("Config","MaxLogFileSize"))
numbackups = int(config.get("Config","NumBackups"))
logfile    = config.get("Config","LogFile")

# Setup logging
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.DEBUG)
handler = logging.handlers.RotatingFileHandler(logfile, maxBytes=backupsize, backupCount=numbackups)
my_logger.addHandler(handler)

# Do the job
my_logger.info("#########################################################################################")
my_logger.info("-----------------------------------------------------------------------------------------")
my_logger.info("Starting backup")
my_logger.info("Time: " + datetime.datetime.now().strftime("%Y-%m-%d %H:%M"))
my_logger.info("-----------------------------------------------------------------------------------------")

# Run individual backups
for jobfolder in folderlist:
    command = rsyncexe + " " + ' '.join(rsyncflags) + " -s \"" + jobfolder + "\" \"" + backuploc + "\""
    my_logger.info("Executing: " + command)
    out = subprocess.check_output(shlex.split(command))
    my_logger.info(out)

my_logger.info("-----------------------------------------------------------------------------------------")
my_logger.info("Backup Complete")
my_logger.info("Time: " + datetime.datetime.now().strftime("%Y-%m-%d %H:%M"))
my_logger.info("-----------------------------------------------------------------------------------------")
my_logger.info("#########################################################################################")

backur can be configure using backur.conf which look like the following:

[Config]
BackupLocation: /media/usb/backup/
LogFile: /var/log/backur.log
MaxLogFileSize: 2097152
NumBackups: 5
RSYNCFlags: "-a",
	        "-v",
	        "--delete"
RSYNC: /usr/bin/rsync

[Folders]
FolderList: "/mnt/Drv1/pics",
	        "/mnt/Drv2/my data",
	        "/mnt/Drv3/docs"

backur can also be downloaded in a zip file. I have written and tested backur only on Linux. However, it should also run on Windows provided that Python is installed on the system. Do let me know about your experiences or issues if you use backur.

Continue Reading

Setting up svn+ssh on Ubuntu Server and Tortoise SVN as client on Windows PC

  1. First you need to setup the basic ssh connectivity using e.g. putty and sshd. This is not the focus of this short report.
  2. On the Windows, use Pageant to cache the ssh key. The target in the “Shortcut to Pageant” should be similar to:
    "C:\Program Files (x86)\PuTTY\pageant.exe" C:\path\private_sshkey
  3. Check again that you can now connect to the server without having to enter your password in the putty.
  4. Install subversion on the server using:
    apt-get install subversion
  5. Create the repositories in a location, let’s say /usr/local/svn.
  6. Make your first repository under svn, lets say proj1. Simply create a folder using mkdir.
  7. Now mark this folder as a repository using:
    svnadmin create /usr/local/svn/proj1
  8. Create a new group called subversion and add yourself and www-data to this group:
    groupadd subversion
    adduser youruid subversion
    adduser root subversion
    adduser www-data subversion
  9. Assign the owner rights of the repositories to the www-data user and the subversion group, as well as make all the subsequently added files belong to the subversion group.
    cd /usr/local/svn
    chown -R www-data:subversion proj1
    chmod -R g+rws proj1
  10. That completes the basic subversion setup. You can check the basic setup by checking-out the newly setup repo locally on the linux server by:
    svn co file:///usr/local/svn/proj1
  11. For the ssh setup, first open the /etc/subversion/config file and uncomment this line:
    ssh = $SVN_SSH ssh -q -o ControlMaster=no
  12. Now open the /usr/local/svn/proj1/conf/svnserve.conf file and make changes according to the following, where realm is whatever authentication realm you want to name it. The realm is not very important as long as you keep it same across different repos where you want to use the same authentication:
    anon-access = none
    auth-access = write
    realm = xxxx
  13. Now go to /etc/init.d and copy the skeleton file to svnserve file. Make appropriate changes in the beginning of  the svnserve file as follows:
    PATH=/sbin:/usr/sbin:/bin:/usr/bin
    DESC="Subversion Server"
    NAME=svnserve
    DAEMON=/usr/bin/$NAME
    DAEMON_ARGS="-d -r /usr/local/svn"
    PIDFILE=/var/run/$NAME.pid
    SCRIPTNAME=/etc/init.d/$NAME
  14. Set the execution bit by: chmod +x svnserve
  15. Now issue the following command to automatically start the svnserve at the bootup:
    update-rc.d svnserve defaults
  16. That is all for the settings on the server side. Start the svnserve using: /etc/init.d/svnserve start
  17. On the windows side, do the following:
    1. Start Pageant and enter the password.
    2. Open Tortoise SVN settings and ask it to use Plink as shown in the image below:
  18. Note that the -l parameter provides the username that should be used for the repository. The ssh key must belong to this user name.
  19. Now checkout the repository by providing the link in the following manner:
    svn+ssh://yourhost/usr/local/svn/proj1
    You must take care to provide the full path to the svn repository. This is because ssh is only tunneling the svn access which means it is similar to accessing it locally on the server itself. Other authentication methods like sasl, etc. allow accessing the repos in a simpler way like svn://yourhost/proj1 but this is not the case with ssh. There might be a way to achieve this also with ssh but I am not aware of it.
  20. That is it. Now you have a functional svn server which can be securely accessed through ssh from anywhere.
Continue Reading

Write your own Python Plugin for Dreambox Newnigma2 and Map to a QuickButton

I have a Samsung TV that is connected to the Dreambox 800SE. The audio output of the dreambox goes to a Sony Blu Ray player. I can turn on the Sony Blu Ray player through Samsung TV remote control but not through Dreambox remote. This always requires also having the Samsung remote handy. What I wanted was to simply map a long press of a quickbutton (e.g., color buttons on the dreambox remote) to a plugin that will send Samsung remote control commands to the Samsung TV. I have written a python script that shows how you can achieve this in another post. The following diagram will explain the setup that I just described.

So actually what I want is to send remote control commands to the Samsung TV through a plugin from the Dreambox which will in turn use my python script to send them to the TV. The TV will then execute them to connect over CEC to the Sony Blu Ray player.

To write the plugin for dreambox, I have used Python. The script development has been explained nicely in this guide which I also used as guideline. The main  component of the plugin is the plugin.py file which for my case is very simple as shown below.

# My own small Plugin
# Author: Asif Iqbal
# Date: 06 APR 2012

from Plugins.Plugin import PluginDescriptor
import os

def main(session, **kwargs):
        os.system('python /etc/samsungremote.py')

def Plugins(**kwargs):
        return PluginDescriptor(
                name="ExecuteMyScript",
                description="Samsung TV Remote",
                where = PluginDescriptor.WHERE_PLUGINMENU,
                fnc=main)

As you can see the plugin itself is very simple. It just calls your python script. The Python script as I have explained above can be found in one of my other posts. To use the plugin, follow these steps:

  • Create a folder according to the name of your plugin (in my case ExecuteMyScript)  /usr/lib/enigma2/python/Plugins/Extensions/
  • Put the above explained plugin script plugin.py under /usr/lib/enigma2/python/Plugins/Extensions/ExecuteMyScript or the folder name that you chose.
  • You also need to place an empty file named __init__.py in this folder.
  • Make sure that the link to your own script is correct in the plugin.py file.
  • Now you need to reboot the Dreambox (or restart GUI).
  • After reboot, use the QuickButton Plugin to choose your Plugin (ExecuteMyScript, in my case) to map to the long press of a color button of your choice.
  • That’s it. You have created your own Dreambox plugin and have successfully mapped to a quick button.

A zip file containing both the plugin files (__ini__.py and the plugin.py) can be found on my download page.

Continue Reading

Samsung TV Remote Control over LAN/WLAN using Python

For quite some time, I have been looking for a method to control my Samsung UE55C8000 TV using Linux scripts. Finally, I found this forum post on SamyGo that provides an example using Perl. As I like to do scripting in Python, so I have translated the script to Python with some minor changes. As you can see in the following Python code, the script contains Key names for all the remote control keys and even some extra. You should be able to easily modify the script to your needs. The script can also be downloaded on my download page.

#!  /usr/bin/python
#   Title: samsungremote.py
#   Author: Asif Iqbal
#   Date: 05APR2012
#   Info: To send remote control commands to the Samsung tv over LAN
#   TODO:

import socket
import base64
import time, datetime

#IP Address of TV
tvip = "100.0.0.123"
#IP Address of TV
myip = "100.0.0.112"
#Used for the access control/validation, but not after that AFAIK
mymac = "00-0c-29-3e-b1-4f"
#What the iPhone app reports
appstring = "iphone..iapp.samsung"
#Might need changing to match your TV type
tvappstring = "iphone.UE55C8000.iapp.samsung"
#What gets reported when it asks for permission
remotename = "Python Samsung Remote" 

# Function to send keys
def sendKey(skey, dataSock, appstring):
 messagepart3 = chr(0x00) + chr(0x00) + chr(0x00) + chr(len(
base64.b64encode(skey))) + chr(0x00) + base64.b64encode(skey);
 part3 = chr(0x00) + chr(len(appstring)) + chr(0x00) \
+ appstring + chr(len(messagepart3)) + chr(0x00) + messagepart3
 dataSock.send(part3);

# Open Socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((tvip, 55000))

# Key Reference
# Normal remote keys
 #KEY_0
 #KEY_1
 #KEY_2
 #KEY_3
 #KEY_4
 #KEY_5
 #KEY_6
 #KEY_7
 #KEY_8
 #KEY_9
 #KEY_UP
 #KEY_DOWN
 #KEY_LEFT
 #KEY_RIGHT
 #KEY_MENU
 #KEY_PRECH
 #KEY_GUIDE
 #KEY_INFO
 #KEY_RETURN
 #KEY_CH_LIST
 #KEY_EXIT
 #KEY_ENTER
 #KEY_SOURCE
 #KEY_AD #KEY_PLAY
 #KEY_PAUSE
 #KEY_MUTE
 #KEY_PICTURE_SIZE
 #KEY_VOLUP
 #KEY_VOLDOWN
 #KEY_TOOLS
 #KEY_POWEROFF
 #KEY_CHUP
 #KEY_CHDOWN
 #KEY_CONTENTS
 #KEY_W_LINK #Media P
 #KEY_RSS #Internet
 #KEY_MTS #Dual
 #KEY_CAPTION #Subt
 #KEY_REWIND
 #KEY_FF
 #KEY_REC
 #KEY_STOP
# Bonus buttons not on the normal remote:
 #KEY_TV
#Don't work/wrong codes:
 #KEY_CONTENT
 #KEY_INTERNET
 #KEY_PC
 #KEY_HDMI1
 #KEY_OFF
 #KEY_POWER
 #KEY_STANDBY
 #KEY_DUAL
 #KEY_SUBT
 #KEY_CHANUP
 #KEY_CHAN_UP
 #KEY_PROGUP
 #KEY_PROG_UP

# First configure the connection
ipencoded = base64.b64encode(myip)
macencoded = base64.b64encode(mymac)
messagepart1 = chr(0x64) + chr(0x00) + chr(len(ipencoded)) \
+ chr(0x00) + ipencoded + chr(len(macencoded)) + chr(0x00) \
+ macencoded + chr(len(base64.b64encode(remotename))) + chr(0x00) \
+ base64.b64encode(remotename)

part1 = chr(0x00) + chr(len(appstring)) + chr(0x00) + appstring \
+ chr(len(messagepart1)) + chr(0x00) + messagepart1
sock.send(part1)

messagepart2 = chr(0xc8) + chr(0x00)
part2 = chr(0x00) + chr(len(appstring)) + chr(0x00) + appstring \
+ chr(len(messagepart2)) + chr(0x00) + messagepart2
sock.send(part2)

# Now send the keys as you like, e.g.,
sendKey("KEY_VOLUP",sock,tvappstring)
time.sleep(1)
sendKey("KEY_TOOLS",sock,tvappstring)
time.sleep(1)
sendKey("KEY_RIGHT",sock,tvappstring)
time.sleep(1)
sendKey("KEY_DOWN",sock,tvappstring)
time.sleep(1)
sendKey("KEY_RIGHT",sock,tvappstring)

# Close the socket when done
sock.close()

 

Continue Reading

Remote Desktop to a Virtual Machine on VirtualBox running on a Headless Server

I have followed the excellent directions given by Falko Timme at howtoforge. I am copying his directions here with some changes to install a Windows 7 system.

First you need to add the VirtualBox repo to the Ubuntu resources.lst:
emacs /etc/apt/sources.list
Add the following line to it:
deb http://download.virtualbox.org/virtualbox/debian oneiric contrib

Then download and install the VirtualBox public key:
wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add -
Update our package database:
apt-get update
Afterwards, install the VirtualBox 4.1 as follows:
apt-get install linux-headers-$(uname -r) build-essential virtualbox-4.1 dkms

Now install the VirtualBox extensions.

cd /tmp
wget http://download.virtualbox.org/virtualbox/4.1.10/Oracle_VM_VirtualBox_Extension_Pack-4.1.10-76795.vbox-extpack
VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.1.8-75467.vbox-extpack

Now add the user who will run VirtualBox to the vboxusers group:
adduser root vboxusers

Now get a Windows 7 iso and do the following steps to install it to a virtual machine:

VBoxManage createvm --name "Windows 7" --register
VBoxManage modifyvm "Windows 7" --memory 2048 --acpi on --boot1 dvd --nic1 bridged --bridgeadapter1 eth0
VBoxManage createhd --filename windows7.vdi --size 50000
VBoxManage storagectl "Windows 7" --name "IDE Controller" --add ide
VBoxManage storageattach "Windows 7" --storagectl "IDE Controller" --port 0 --device 0 --type hdd --medium /mnt/Windows/windows7.vdi
VBoxManage storageattach "Windows 7" --storagectl "IDE Controller" --port 1 --device 0 --type dvddrive --medium /mnt/Windows/Windows7Ultimate.iso
VBoxHeadless --startvm "Windows 7"

You can now connect using Remote Desktop to this virtual machine.

Continue Reading

Useful tmux Commands

tmux is a powerful console window manager. The most valuable feature of tmux is the ability to detach from a running tmux session, exit the ssh session, relogin to ssh again and resume the tmux session. This is simply beautiful. Here I list some handy tmux commands.

  • tmux
    starts the tmux.
  • ^b d
    detaches from a running tmux session.
  • tmux attach
    attaches to a running tmux session.
  • ^b ?
    lists all the key-bindings. Great help.
  • ^b %
    splits the window into two vertical panes.
  • ^b c
    creates a new window.
  • ^b RIGHT/LEFT
    switches to right/left panes.
  • ^b PGUP/PGDOWN
    scrolls the buffer up/down.
  • ^b d
    detaches the current session.
  • tmux attach -d
    attaches to a tmux session, detaching the other clients.

Thank you tmux-team for developing such a useful program.

Continue Reading