Let's make the world a better place using digital technologies!

Category: Linux

Setting Up SVN on Local Network

Assuming svn is installed on both computers (use apt-get install subversion)

On the Server:

  • Create the root repository container folder, e.g., /usr/src/my_data/svnrepos
  • Create the user that you’ll use to access the repository remotely.
  • adduser --no-create-home --ingroup subversion rpi (select passowrd. Create group subversion beforehand if needed)
  • smbpasswd -a rpi(use the same password if you like)
  • chown -R root:subversion svnrepos (chown to root and subversion group where the user should also belong)
  • Share the above folder using samba with the “force group” and “valid users” commands. Also change the masks to 0775.
  • Create svn respositry: svnadmin create raspi
  • Share top svn folder (/usr/src/my_data/svnrepos) using samba

On the Client:

  • Mount shared folder under, e.g., /mnt/svnrepos using, e.g.:
    //192.168.0.10/svnrepos /mnt/svnrepos cifs user=rpi,pass=password 0 0
  • Mount it using:
    mount -a
  • Go to where you want to create the repositorycheckout the respository, e.g. /usr/src
  • Check out the repository using:
    svn checkout file:///mnt/svnrepos/raspi
Continue Reading

Installing a new ssl Certificate on your own Linux Server

I use startssl as they provide an excellent free ssl certificate for private use. You can choose any that you like.

  • Register on startssl and start domain validation process
  • Once, the domain is validated, e.g., via email, follow these steps to create an ssl CSR (certificate request)
    openssl req -newkey rsa:2048 -keyout yourdomainname.key -out yourdomainname.csr
  • Note: the key and csr files can be named as you like.
  • Unencrypt the yourdomainname.key file using the following command:
    openssl rsa -in yourdomainname.key -out yourdomanprivate.key
  • Copy the yourdomainprivate.key file to your Apache installation (e.g., /etc/apache2/ssl)
  • Start the process of ssl certificate creation on startssl and paste your csr in the box provided.
  • Download the generated zip file and unzip to find the Apache (or whatever server your are using) version.
  • copy the 1_root_bundle.crt and yourdomainname.crt files to your apache ssl directory (e.g., /etc/apache2/ssl)
  • Update the key names in the sites-enabled config file like:
    SSLCertificateFile /etc/apache2/ssl/yourdomainname.crt                           
    SSLCertificateKeyFile /etc/apache2/ssl/yourdomainprivate.key                        
    SSLCertificateChainFile /etc/apache2/ssl/1_root_bundle.crt
  • Restart Apache by:
    service apache2 restart
  • Note: Even when you are renewing with startssl, the process is the same as first time certificate creation.
Continue Reading

Using sshfs to mount Linux shares on Windows over ssh/Putty

For really long time, I’d been looking for a simpler way to access Linux shares on Windows over ssh. It’s tedious process to map them via Samba/CIFS driver on Windows that requires setting up a virtual network adapter to route the requests via Putty to the remote server. In enterprise environment, you’ve issues with this approach when the sharing is combined with local network shares.

Fortunately, some geeky souls have been working on this problem and we now have a good solution. I’m particulary thankful to the maintainers of the following projects:

Dokany is a windows file driver that provides file system functions to a file system application.
WinSshFS is a windows application that, together with Dokany, maps a remote directory over ssh as a local windows drive.

Steps:

  • Configure ssh access on Linux
  • Confirm that it works using Putty on Windows
  • Configure Pageant with the ssh private key of the Linux user who will be accessing the share
  • Make sure that user has access to the share on Linux
  • Download and install Dokany driver on Windows
  • Download and run WinSshFS
  • Map the drive in the WinSshFS, see image below for hint.

Important: Make sure you use mutually compatible versions of both Dokany and WinSshFS.

Continue Reading

Share Data using FTP Server

First create an appropriate user, with no shell access, belonging to ftp-users group and chrooted jail at /srv/ftp/xxxx:

useradd ftp-userxx -G ftp-users -s /bin/false -d /srv/ftp/xxxx

Now assign a password to the user, and make it the owner of the shared folder:
passwd ftp-userxx
chown -R ftp-userxx /srv/ftp/xxxx

Now copy the files to be shared into the respective folder. After that change the permissions to read-only. Be careful! we need +x in order to be able to do chroot:
chmod -R 544 /srv/ftp/xxxx

Assuming we are doing it using proftp, the following should be given:

  • proftpd running
  • Port 21 open
  • In /etc/proftpd/proftpd.conf file: RequireValidShell off
  • And to keep the users in the chrooted jail: DefaultRoot ~
Continue Reading

Reset USB 3.0 Device from the Linux Shell

If you are getting errors like:
cdc_acm 3-11:1.2: acm_submit_read_urb - usb_submit_urb failed: -12
or
xhci_hcd 0000:00:14.0: ERROR no room on ep ring
or
tty_port_close_start: tty->count = 1 port count = 0.

then most probably xhci (USB3) module is causing this problem. Generally you can fix this problem (at least temporarily) by removing and re-inserting the USB connector. However, it would be much better to do it from the command line. It is possible by doing:
echo -n "0000:00:14.0" | tee /sys/bus/pci/drivers/xhci_hcd/unbind
and then
echo -n "0000:00:14.0" | tee /sys/bus/pci/drivers/xhci_hcd/bind

Note that “0000:00:14.0” is the id of the malfunctioning USB device that can be different in your case. You can easily find this by inspecting your log (use dmesg).

Continue Reading

php does not show any output

If the php from the linux cli is not showing any output, then most probably it is crashing with a fatal error. It will not show any output if the errors are turned off in /etc/php5/cli/php.ini.

Turn on the errors by setting display_errors = On and you will find out what is causing the problem

Note: If php cli is not working because of a bad configuration, then pear will also not show any output.

Continue Reading

Using svn+ssh from Eclipse through Pageant & Plink under Windows 7 64-bit

So I just want to use my svn repositories that are already configured and ready to use with svn+ssh protocol. I can check them out through TortoiseSVN as I have explained in one of my previous posts. Now the next step is to integrate it directly in the Eclipse IDE. Here are the steps needed to achieve that:

  • Add new site http://subclipse.tigris.org/update_1.8.x through Help->Install New Software in Eclipse.
  • Install both Subclipse and SVNKit.
  • Create a new system variable SVN_SSH with the following value:
    C:\\Program Files\\TortoiseSVN\\bin\\TortoisePlink.exe -ssh -2 -A -l username.
    Take care to match the path and the username to your needs.
  • Restart Eclipse
  • Go to Window->Show View->Other and choose SVN Repositories.

  • Now go in the SVN Repositories tab on the lower side of the IDE and add the path to your repository (svn+ssh://yoursite/repo). If the authentication is working properly, you will now see you remote repository:
  • Now you can right-click on the desired project and Checkout to Eclipse to work on it. All the usual SVN commands will be available through Team menu.
Continue Reading

backur: A Configurable Python Utility for Optimized Incremental Backups

I have been on the lookout for an efficient small utility to incrementally backup my Linux server to an external drive. I have looked at various options like Back in Time and Backup PC. They are however too advanced and too automated for my purpose. My requirements are the following:

  1. Incremental backups
  2. Console based
  3. Preservation of file times, permissions, user and group
  4. Easy configuration
  5. Logging support
  6. Mirror mode

I have been using rsync for some time in bash scripts. As Python is becoming more and more my language of scripting, so I wanted to write it in Python. With that background information, here comes “backur”. backur is very easy to configure and use. backur.conf, as given below, is self-explanatory. It also provides customizable logging including log-file rotation. Th backup and source paths are very easy to enter, and allow spaces, thanks to shlex split method. You can run backur either manually, or through as a cron job, as it suits you.

#! /usr/bin/python

# Title:    backur.py
# Author:   Asif Iqbal (aiqbalrana@gmail.com)
# Date:     11OCT2012
# Info:     "backur" can be used to automatically backup given folder locations
#           to another backup location. backur is easily configurable through a
#           configuration file. backur uses rsync to achieve highly optimized
#           data transfers by doing only the incremental updates.    

import ConfigParser
import re, os
import glob
import logging
import logging.handlers
import subprocess
import datetime
import shlex

def splitNStripArgs(inputStr, charToStrip):
    data = inputStr.split(',')
    retData = []
    for item in data:
        retData.append((item.strip()).strip(charToStrip))
    return retData

# Read configuration data
confpath = os.path.dirname(__file__)
config = ConfigParser.ConfigParser()
config.read(confpath + "/backur.conf")

rsyncflags = splitNStripArgs((config.get("Config", "RSYNCFlags")),'"')
rsyncexe   = config.get("Config", "RSYNC")
folderlist = splitNStripArgs((config.get("Folders", "FolderList")),'"')
backuploc  = config.get("Config","BackupLocation")
backupsize = int(config.get("Config","MaxLogFileSize"))
numbackups = int(config.get("Config","NumBackups"))
logfile    = config.get("Config","LogFile")

# Setup logging
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.DEBUG)
handler = logging.handlers.RotatingFileHandler(logfile, maxBytes=backupsize, backupCount=numbackups)
my_logger.addHandler(handler)

# Do the job
my_logger.info("#########################################################################################")
my_logger.info("-----------------------------------------------------------------------------------------")
my_logger.info("Starting backup")
my_logger.info("Time: " + datetime.datetime.now().strftime("%Y-%m-%d %H:%M"))
my_logger.info("-----------------------------------------------------------------------------------------")

# Run individual backups
for jobfolder in folderlist:
    command = rsyncexe + " " + ' '.join(rsyncflags) + " -s \"" + jobfolder + "\" \"" + backuploc + "\""
    my_logger.info("Executing: " + command)
    out = subprocess.check_output(shlex.split(command))
    my_logger.info(out)

my_logger.info("-----------------------------------------------------------------------------------------")
my_logger.info("Backup Complete")
my_logger.info("Time: " + datetime.datetime.now().strftime("%Y-%m-%d %H:%M"))
my_logger.info("-----------------------------------------------------------------------------------------")
my_logger.info("#########################################################################################")

backur can be configure using backur.conf which look like the following:

[Config]
BackupLocation: /media/usb/backup/
LogFile: /var/log/backur.log
MaxLogFileSize: 2097152
NumBackups: 5
RSYNCFlags: "-a",
	        "-v",
	        "--delete"
RSYNC: /usr/bin/rsync

[Folders]
FolderList: "/mnt/Drv1/pics",
	        "/mnt/Drv2/my data",
	        "/mnt/Drv3/docs"

backur can also be downloaded in a zip file. I have written and tested backur only on Linux. However, it should also run on Windows provided that Python is installed on the system. Do let me know about your experiences or issues if you use backur.

Continue Reading

Setting up svn+ssh on Ubuntu Server and Tortoise SVN as client on Windows PC

  1. First you need to setup the basic ssh connectivity using e.g. putty and sshd. This is not the focus of this short report.
  2. On the Windows, use Pageant to cache the ssh key. The target in the “Shortcut to Pageant” should be similar to:
    "C:\Program Files (x86)\PuTTY\pageant.exe" C:\path\private_sshkey
  3. Check again that you can now connect to the server without having to enter your password in the putty.
  4. Install subversion on the server using:
    apt-get install subversion
  5. Create the repositories in a location, let’s say /usr/local/svn.
  6. Make your first repository under svn, lets say proj1. Simply create a folder using mkdir.
  7. Now mark this folder as a repository using:
    svnadmin create /usr/local/svn/proj1
  8. Create a new group called subversion and add yourself and www-data to this group:
    groupadd subversion
    adduser youruid subversion
    adduser root subversion
    adduser www-data subversion
  9. Assign the owner rights of the repositories to the www-data user and the subversion group, as well as make all the subsequently added files belong to the subversion group.
    cd /usr/local/svn
    chown -R www-data:subversion proj1
    chmod -R g+rws proj1
  10. That completes the basic subversion setup. You can check the basic setup by checking-out the newly setup repo locally on the linux server by:
    svn co file:///usr/local/svn/proj1
  11. For the ssh setup, first open the /etc/subversion/config file and uncomment this line:
    ssh = $SVN_SSH ssh -q -o ControlMaster=no
  12. Now open the /usr/local/svn/proj1/conf/svnserve.conf file and make changes according to the following, where realm is whatever authentication realm you want to name it. The realm is not very important as long as you keep it same across different repos where you want to use the same authentication:
    anon-access = none
    auth-access = write
    realm = xxxx
  13. Now go to /etc/init.d and copy the skeleton file to svnserve file. Make appropriate changes in the beginning of  the svnserve file as follows:
    PATH=/sbin:/usr/sbin:/bin:/usr/bin
    DESC="Subversion Server"
    NAME=svnserve
    DAEMON=/usr/bin/$NAME
    DAEMON_ARGS="-d -r /usr/local/svn"
    PIDFILE=/var/run/$NAME.pid
    SCRIPTNAME=/etc/init.d/$NAME
  14. Set the execution bit by: chmod +x svnserve
  15. Now issue the following command to automatically start the svnserve at the bootup:
    update-rc.d svnserve defaults
  16. That is all for the settings on the server side. Start the svnserve using: /etc/init.d/svnserve start
  17. On the windows side, do the following:
    1. Start Pageant and enter the password.
    2. Open Tortoise SVN settings and ask it to use Plink as shown in the image below:
  18. Note that the -l parameter provides the username that should be used for the repository. The ssh key must belong to this user name.
  19. Now checkout the repository by providing the link in the following manner:
    svn+ssh://yourhost/usr/local/svn/proj1
    You must take care to provide the full path to the svn repository. This is because ssh is only tunneling the svn access which means it is similar to accessing it locally on the server itself. Other authentication methods like sasl, etc. allow accessing the repos in a simpler way like svn://yourhost/proj1 but this is not the case with ssh. There might be a way to achieve this also with ssh but I am not aware of it.
  20. That is it. Now you have a functional svn server which can be securely accessed through ssh from anywhere.
Continue Reading

Write your own Python Plugin for Dreambox Newnigma2 and Map to a QuickButton

I have a Samsung TV that is connected to the Dreambox 800SE. The audio output of the dreambox goes to a Sony Blu Ray player. I can turn on the Sony Blu Ray player through Samsung TV remote control but not through Dreambox remote. This always requires also having the Samsung remote handy. What I wanted was to simply map a long press of a quickbutton (e.g., color buttons on the dreambox remote) to a plugin that will send Samsung remote control commands to the Samsung TV. I have written a python script that shows how you can achieve this in another post. The following diagram will explain the setup that I just described.

So actually what I want is to send remote control commands to the Samsung TV through a plugin from the Dreambox which will in turn use my python script to send them to the TV. The TV will then execute them to connect over CEC to the Sony Blu Ray player.

To write the plugin for dreambox, I have used Python. The script development has been explained nicely in this guide which I also used as guideline. The main  component of the plugin is the plugin.py file which for my case is very simple as shown below.

# My own small Plugin
# Author: Asif Iqbal
# Date: 06 APR 2012

from Plugins.Plugin import PluginDescriptor
import os

def main(session, **kwargs):
        os.system('python /etc/samsungremote.py')

def Plugins(**kwargs):
        return PluginDescriptor(
                name="ExecuteMyScript",
                description="Samsung TV Remote",
                where = PluginDescriptor.WHERE_PLUGINMENU,
                fnc=main)

As you can see the plugin itself is very simple. It just calls your python script. The Python script as I have explained above can be found in one of my other posts. To use the plugin, follow these steps:

  • Create a folder according to the name of your plugin (in my case ExecuteMyScript)  /usr/lib/enigma2/python/Plugins/Extensions/
  • Put the above explained plugin script plugin.py under /usr/lib/enigma2/python/Plugins/Extensions/ExecuteMyScript or the folder name that you chose.
  • You also need to place an empty file named __init__.py in this folder.
  • Make sure that the link to your own script is correct in the plugin.py file.
  • Now you need to reboot the Dreambox (or restart GUI).
  • After reboot, use the QuickButton Plugin to choose your Plugin (ExecuteMyScript, in my case) to map to the long press of a color button of your choice.
  • That’s it. You have created your own Dreambox plugin and have successfully mapped to a quick button.

A zip file containing both the plugin files (__ini__.py and the plugin.py) can be found on my download page.

Continue Reading