Sending email via Gmail in Django

While I was learning how to use django-registration application, I need to know that how to send an email for the user to activate the registration.
During this post I want to show how to test it using shell, 

1. Create a project, django-admin.py startproject pjtname
2. Edit settings.py with code below:

         EMAIL_USE_TLS =True
         EMAIL_HOST = 'smtp.gmail.com'
         EMAIL_HOST_USER = 'youremail@gmail.com'
         EMAIL_HOST_PASSWORD = 'your password'
         EMAIL_PORT = 587
     3. Run interactive mode, python manage.py shell
     4. Import the EmailMessage module,
         from django.core.mail import EmailMessage
     5. Send the email,
         email = EmailMessage ('Subject', 'Body', to=['receiver email id'])
         email.send()

Asynchronous Socket Programming in Python

How to transfer data from one client to another through a server? The solution is very easy using the select call supported in sockets.

To understand the benefit of select call in sockets, one should understand asynchronous socket programming. Assume a case where many clients connect to a server and send data for processing concurrently, then the server has to handle the clients asynchronously.

In synchronous socket programming, the server processes each client sequentially, in this case when it waits for a response/data from a client using the recv call, it blocks or in other words the recv call cannot return until there is some data received from the socket, in a real time scenario, this way of handling clients is inefficient in the sense that all other connected clients need to wait till the server completes processing the current one.

Therefore one needs a more elegant way to asynchronously handle client requests or the ability to read, write from multiple sockets whenever they are ready to be read or written, which is where the select call comes handy.

To explain the use of select system call, I will illustrate a TCP/IP chat Client server program, where the functionalities of the server and the client program were mentioned below.

TCP/IP Chat Server:

1. Accepts connection from multiple clients

2. Use select call to get the list of available sockets which are ready to be read.

3. Whenever a client connects, the server notifies all other connected clients of this new connection, in the same way the server notifies all when a client quits or that client connection is lost.

4. The server broadcasts data sent by a client to all other connected clients.

TCP/IP Chat Client:

1. Connects to the server and starts two threads, one to process received data and one for getting data input to be sent to other connected clients through the server.

2. When the client quits (using q or Q) or the server is suddenly down (handle the worst case scenario), the socket is closed and the process exits.

3. Whenever any thread closes the socket connection, it interrupts the main program using the thread.interrupt_main() call, then the main exits.

The syntax used for the select call is as follows

read_sockets,write_sockets,error_sockets = select.select(CONNECTION_LIST,[],[])

The select call returns three lists, the list of sockets which are ready to be read, written and those which caused an error, since we are interested only in the list of sockets which are ready to be read, we use only a single select input parameter.

The client and server Chat program were implemented in python for the ease of understanding. Try to see how the chat client/server works by launching multiple clients in different windows connected to the chat server.

Server.py

# The server accepts connection from multiple clients and
# broadcasts data sent by a client to all other clients
# which are online (connection active with server)

import socket
import select
import string

def broadcast_data (sock, message):
    """Send broadcast message to all clients other than the
       server socket and the client socket from which the data is received."""
    
    for socket in CONNECTION_LIST:
        if socket != server_socket and socket != sock:            
            socket.send(message)

if __name__ == "__main__":

    # List to keep track of socket descriptors
    CONNECTION_LIST=[]

    # Do basic steps for server like create, bind and listening on the socket
    
    server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server_socket.bind(("127.0.0.1", 5000))
    server_socket.listen(10)

    # Add server socket to the list of readable connections
    CONNECTION_LIST.append(server_socket)

    print "TCP/IP Chat server process started."

    while 1:
        # Get the list sockets which are ready to be read through select
        read_sockets,write_sockets,error_sockets = select.select(CONNECTION_LIST,[],[])

        for sock in read_sockets:

            if sock == server_socket:
                # Handle the case in which there is a new connection recieved
                # through server_socket
                sockfd, addr = server_socket.accept()
                CONNECTION_LIST.append(sockfd)
                print "Client (%s, %s) connected" % addr
                broadcast_data(sockfd, "Client (%s, %s) connected" % addr)

            else:
                # Data recieved from client, process it
                try:
                    #In Windows, sometimes when a TCP program closes abruptly,
                    # a "Connection reset by peer" exception will be thrown
                    data = sock.recv(4096)
                except:
                    broadcast_data(sock, "Client (%s, %s) is offline" % addr)
                    print "Client (%s, %s) is offline" % addr
                    sock.close()
                    CONNECTION_LIST.remove(sock)
                    continue

                if data:
                    # The client sends some valid data, process it
                    if data == "q" or data == "Q":
                        broadcast_data(sock, "Client (%s, %s) quits" % addr)
                        print "Client (%s, %s) quits" % addr
                        sock.close()
                        CONNECTION_LIST.remove(sock)
                    else:
                        broadcast_data(sock, data)                       
                
    server_socket.close()    

client.py

# The client program connects to server and sends data to other connected 
# clients through the server
import socket
import thread
import sys

def recv_data():
    "Receive data from other clients connected to server"
    while 1:
        try:
            recv_data = client_socket.recv(4096)            
        except:
            #Handle the case when server process terminates
            print "Server closed connection, thread exiting."
            thread.interrupt_main()
            break
        if not recv_data:
                # Recv with no data, server closed connection
                print "Server closed connection, thread exiting."
                thread.interrupt_main()
                break
        else:
                print "Received data: ", recv_data

def send_data():
    "Send data from other clients connected to server"
    while 1:
        send_data = str(raw_input("Enter data to send (q or Q to quit):"))
        if send_data == "q" or send_data == "Q":
            client_socket.send(send_data)
            thread.interrupt_main()
            break
        else:
            client_socket.send(send_data)
        
if __name__ == "__main__":

    print "*******TCP/IP Chat client program********"
    print "Connecting to server at 127.0.0.1:5000"

    client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    client_socket.connect(('127.0.0.1', 5000))

    print "Connected to server at 127.0.0.1:5000"

    thread.start_new_thread(recv_data,())
    thread.start_new_thread(send_data,())

    try:
        while 1:
            continue
    except:
        print "Client program quits...."
        client_socket.close()       

How to strip of mulitiple white spaces.

During this post, I want to share some different solutions for stripping multiple white spaces in the  sentence.
We need to strip the multiple spaces betweens the words. look the following solution. 

Example:

>>> str='Multiple      white  spaces'
>>> for i in str.split():
...     print i,
... 
Multiple white spaces

you can strip off trailing and leading whitespaces using strip(). So there's no need for lstrip,rstrip
Example:

>>> str='    multiple white     spaces    '
>>> str.strip()
'multiple white     spaces'

If you know the number of multiple white spaces in the middle, example, 2 white spaces, you can use replace() , example for 2 white spaces:

Example:

>>> str='How  to strip    multiple  white  spaces'
>>> str.replace("  "," ")
'How to strip  multiple white spaces'

Example for stripping multiple white spaces :

>>> str='How  to strip    multiple  white  spaces'
>>> ' '.join(str.split())
'How to strip multiple white spaces' 

__repr__ in python

_repr__ is used to print a human readable presentation of an object. In this case it prints the class name and some other attributes. A simple example:

>>> class point:
...     def __init__(self,x,y):
...             self.x,self.y=x,y
...     def __repr__(self):
...             return 'point(x=%s, y=%s)' %(self.x,self.y)
... 
>>> p=point(1,2)
>>> p
point(x=1, y=2)

String reverse in Python

During this post I want share an interesting string reverse solution in python. Here is the question.

Write a simple program that reads a line from the keyboard and outputs the same line where every word is reversed. A word is defined as a continuous sequence of alphanumeric characters or hyphen (‘-’). For instance, if the input is

“We are at Ignite Solutions!”

the output should be

“eW era ta etingI snoituloS!”

At first, I just tried with the following simple code,

    print"Enter the string:"
    str1=raw_input()
    print (' '.join((str1[::-1]).split(' ')[::-1])) 

It prints the outputs like, eW era ta etingI !snoituloS. Is it correct??

At first look we thought its a correct answer, but actually it is  wrong. Can you got the problem?

The real problem is in the position of exclamation(!).

I have found one new solution for this problem, please listen here.

    str1=raw_input("Enter the string: ")
    #re.sub() used to find each word and reverse it
    print re.sub(r'[-\w]+', lambda w:w.group()[::-1],str1)

It prints the output like this, eW era ta etingI snoituloS!

os.path.walk python example

Last week, I was interviewed by ignite solution recruitment team, I really enjoyed every part of it. During the interview process, I was needed to attend an online test. There are only three more questions on the test. I want to share one of the interesting question on the test with this blog post. Here is the question,

Write a program that accepts a string on the command line and prints out the list of files within a folder (or subfolders) that have the string within them.

Solution:

import os,re,sys
def fun(arg, dirname, fnmes):
    out_list = []
    for fname in fnmes:
        filepath = os.path.join(dirname,fname)
        if os.path.isfile(filepath):
            fp = open(filepath ,'r')
            text = fp.read()
            fp.close()
            if re.search(arg,text):
                # print full path. Calling os.path.basename will
                # give us just the name.
                print filepath

def main():
    strg=sys.argv[1]
    os.path.walk('.', fun, strg)

if __name__=='__main__':
    main()

Here is a simple os.path.walk() example which walks your directory tree and returns the list of file names that contains the given string.

Find the Union and Intersection of two lists in python

Here are three functions using sets to remove duplicate entries from a list, find the intersection of two lists, and find the union of two lists. Note, sets were introduced in Python 2.4, so Python 2.4 or later is required.

def unique(a):
    """ return the list with duplicate elements removed """
    return list(set(a))

def intersect(a, b):
    """ return the intersection of two lists """
    return list(set(a) & set(b))

def union(a, b):
    """ return the union of two lists """
    return list(set(a) | set(b))

if __name__ == "__main__":
    a = [1,2,3,4,3,2,1,0,9,8,7,6,5,4,3,1]
    b = [19,18,17,12,1,4,5,6,13,14,15,13,16,20]
    print unique(a)
    print unique(b)
    print intersect(a, b)
    print union(a, b)

Output

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[1, 4, 5, 6, 12, 13, 14, 15, 16, 17, 18, 19, 20]
[1, 4, 5, 6]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20] 

For more information on Python sets, see the Library Reference.

List comprehensions

    Python supports a concept called 'list comprehensions'. It can be used to construct lists in a very natural and easy way. The following are common ways to describe lists (or sets,or tuples, or vectors) in mathematics

     S = {x²: x in {0...9}}

     V = (1,2,4,8,...,2¹²)

    M = {x | x in S and x even }

In python we can written  it as,

>>> S=[x**2 for x in range(10)]

>>> V=[2**i for i in range(13)]

>>> M=[x for x in S if x%2==0]

Output

>>> print S; print V; print M

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096]

[0, 4, 16, 36, 64]

Examples

>>> vec=[2,4,6]

>>> [3*x for x in vec]

[6, 12, 18]

>>> [3*x for x in vec if x>3]

[12, 18]

>>> [3*x for x in vec if x<2]

[]

>>> [[x,x**2] for x in vec]

[[2, 4], [4, 16], [6, 36]]

>>> [x,x**2 for x in vec] # error - need () for tuples

File "<stdin>", line 1

[x,x**2 for x in vec]

^

SyntaxError: invalid syntax

>>> [(x,x**2) for x in vec ]

[(2, 4), (4, 16), (6, 36)]

>>> vec1=[2,4,6]

>>> vec2=[4,3,-9]

>>> [x*y for x in vec1 for y in vec2]

[8, 6, -18, 16, 12, -36, 24, 18, -54]

>>> [x+y for x in vec1 for y in vec2]

[6, 5, -7, 8, 7, -5, 10, 9, -3]

>>> [vec1[i]*vec2[i] for i in range(len(vec1))]

[8, 12, -54]

List comprehensions are more flexible than map() and can be applied to complex expressions and nested functions.

Example

>>> [str(round(355/113.0,i)) for i in range(1,6)]

['3.1', '3.14', '3.142', '3.1416', '3.14159']

In list comprehension list contain any type of elements including strings, nested lists and functions

Example

>>> words='The quick brown fox jumps over the lazy dog'.split()

>>> print words

['The', 'quick', 'brown', 'fox', 'jumps', 'over', 'the', 'lazy', 'dog']

>>>

>>> stuff=[[w.upper(),w.lower(),len(w)] for w in words]

>>> for i in stuff:

... print i

...

['THE', 'the', 3]

['QUICK', 'quick', 5]

['BROWN', 'brown', 5]

['FOX', 'fox', 3]

['JUMPS', 'jumps', 5]

['OVER', 'over', 4]

['THE', 'the', 3]

['LAZY', 'lazy', 4]

['DOG', 'dog', 3]

>>>

>>> stuff=map(lambda w: [w.upper(),w.lower(),len(w)],words)

>>> for i in stuff:

... print i

...

['THE', 'the', 3]

['QUICK', 'quick', 5]

['BROWN', 'brown', 5]

['FOX', 'fox', 3]

['JUMPS', 'jumps', 5]

['OVER', 'over', 4]

['THE', 'the', 3]

['LAZY', 'lazy', 4]

['DOG', 'dog', 3]

This example shows that, we can do the same thing by using the map() and lambda function. But in most cases we use the list comprehension, because it is more efficient and easier to read.

Exception and Exception handling in python

    Exceptions are general programming technique for altering the regular program flow in special cases such as errors or incorrect input.

     Like many other programming languages, python has exception handling via try...except blocks. Python uses try ... except to handle exceptions and raise to generate them. Java and C++ use try ...catch to handle exceptions, and throw to generate them. 

Exceptions

Exceptions are errors, that are detected during the execution of code. In this post I want to explain how we can handle this types of exceptions. Most exceptions are not handled by programs. In such situations we can generate an error message.

Various types of exceptions are ZeroDivisionError,NameError and TypeError.

Example for ZeroDivisionError

>>> 20*(2/0)

Output

ZeroDivisionError: integer division or modulo by zero

Here we try to divide 2 by zero, this is an example for ZeroDivisionError.

Example for NameError

>>> 5+spam*3

Output

NameError: name 'spam' is not defined

Example for TyeError

>>> '6'+3

Output

TypeError: cannot concatenate 'str' and 'int' objects

Exception Handling

    We can catch or handle an exception which a piece of code may generate by putting that piece of code inside of a try block and then specifying corresponding except block.

Example

>>> try:

... some_func()

... except:

... print 'Caught an exception in "some_func()".Exiting.'

... exit(1)

...

Caught an exception in "some_func()".Exiting.

In this example function 'some_func()' generate an error, here we write this function in try block, corresponding solutions are write in the except block.

Multiple except blocks may be associated with a given try block. The first except which matches the type of exception raised will be executed.

Example

>>> try:

... some_func()

... except FoolException,e:

... print 'Caught a FoolException. Exiting.'

... exit(1)

... except RuntimeError,e:

... print 'Caught a RuntimeError. Exiting.'

... exit(10)

... except:

... print 'caught an exception in "some_func()". Exiting'

... exit(100)

Raising Exceptions

The raise statement allows the programmer to force a specified exception to occur.

Example

>>> foo=56

>>> if foo==56: raise RuntimeError("Can't continue - foo is 56")

Output

RuntimeError: Can't continue - foo is 56

finally

There may be some code (like closing a file) that you need to execute no matter whether the code before that raises an exception or runs smoothly. We can put such code in finally block.

Example

>>> fname='file-name.txt'

>>> try:

... f=file(fname)

... except IOError:

... print "File %s could not be opened for reading. Closing."% fname

... exit(1)

>>> try:

... process_file(f)

... finally:

... f.close()

In this example first uses a try block to safely open a file for reading and then uses a try and finally to ensure that the file gets closed.

Functional programming in python

     In python we can pass functions as an argument to another function. This is called higher order functions. Lets define two functions at the interactive python prompt and try some experiments.

>>> def sqr(x): return x*x
...

>>> def cube(x): return x*x*x

... 

>>> def compose(f,g,x): return f(g(x))

...

>>> compose(sqr,cube,3)

729

    In this example we define two functions sqr() and cube(), then we pass these functions as an argument to the another function compose(). Compose function perform the computstion and return the result.

    The functional programming paradigm encourages us to structure our programs as collections of pure functions. The basic element of Functional programming in python are the functions map(),reduce(),and filter() , and the operator lambda.

Lambda

The lambda keyword is used for creating anonymous functions. The body of the lambda should be composed of simple expressions only. More than one arguments are supported, these are separated by commas. Lambda is often used in conjunction with map(),filter() and reduce().

Syntax:

lambda argument_list:expression

Example:

>>> f=lambda x:x+4

>>> f(3)

7

In the above example, we use lambda to create a function which accepts an argument and returns it after adding four.

>>> f=lambda x,y:x+y

>>> f(5,4)

9

    In this example lambda take two arguments and return result after adding the values.

filter()

Filter function accepts a function and a list as arguments and returns the list composed of only those elements for which the function returns true.

syntax:

filter(function,sequence)

Example:

>>> a=[2,3,6,7,11]

>>> output=filter(lambda x:x%2,a)

>>> print output

[3, 7, 11]

Here function 'lambda' will be applied to every element of the list a. If the function return true for a particular list of elements, that will be included in the result list.

map()

Map function accepts a function and a list as arguments and returns the list obtained by applying the function on each element of the original list.

Syntax:

map(function,sequence)

Example

>>> def sqrt(x): return x*x

...

>>> map(sqrt,[2,6,3,1,9])

[4, 36, 9, 1, 81]

     map function accept a function sqrt and a list of elements, then it apply the function to each element of the list and retun the result list.

This example can also be written by using lambda.

>>> map(lambda x:x*x,[2,6,3,1,9])

[4, 36, 9, 1, 81]

reduce()

     Important thing is that, reduce() returns only a single value. It also accept a function and a list of elements, at first it take the first two elements in the list and applied it to the function, in the next step the result of the previous step and the next element is applied to the function, this process will continue at the end of the list, finally a single value returned.

Syntax:

reduce(function,sequence)

Consider the following example which uses reduce to compute the sum of a list of numbers.

Example

>>> nums=[1,2,3,10,20,30]

>>> def add(x,y): return x+y

...

>>> s=reduce(add,nums)

>>> print s

66

map – reduce

Note that the return value of a map is a list while the input of reduce is a list. Thus, the two functions can be combined to a great effect.

Example

>>> a=[1,2,3,10,200,3000]

>>> b=[20,30,40,10,3,2]

>>> def add(x,y):

... return x+y

...

>>> def mul(x,y):

... return x*y

...

>>> result=reduce(add,map(mul,a,b))

>>> print result

6900

     We don't actually have to define the add and mul function above. We can simply use operator.add and operator.mul from the operator module.

Example

>>> import operator

>>> A=[1,2,3,10,200,3000]

>>> B=[20,30,40,10,3,2]

>>> s=reduce(operator.add,map(operator.mul,A,B))

>>> print s

6900

Function

     In this post I want to share some basic concepts about the functions in python. I think this post is very useful for all who want to know the function in python.

     Functions are reusable pieces of programs. They allow you to give name to a block of statements and you can run that block using that name anywhere in your program and any number of times. Functions are defined using the def keyword. This keyword followed by the function name followed by a pair of parentheses which may enclose some names of variables and the lines ends with a colon. Next follows the block of statements that are part of this function.

Example:

def hello():

     print 'Hello World.'

Output

hello() # call the function

Hello World

Parameters

Parameters are specified within the pair of parentheses in the function definition, separated by commas. When we call the function, we supply the values in the same way. The names given in the function definition are called parameters. Whereas the values you supply in the function are called arguments.

Example1

>>> def max(a,b):

...         if a>b:

...             print a,' is maximum'

...         else:

...             print b,' is maximum'

Output

>>> max(7,5) # Directly give the literal values

7 is maximum

Example 2

>>> def max(a,b):

... if a>b:

... print a,' is maximum'

... else:

... print b,' is maximum'

Output

>>> x=5

>>> y=7

>>> max(x,y) # Give variables as arguments

7 is maximum

Local Variable

     Variables defined in the functions are not related to any other variables defined with the same name outside the function. That is the variable names are local to the function.

Example

>>> def fun(x):

...         print 'x is',x

...         x=5

...         print 'changed local x to',x

Output

>>> x=10

>>> fun(x)

x is 10

changed local x to 5

>>> print 'x is still',x

x is still 10

Global statement

    Global statements are defined using the global keyword. It is impossible to assign a value to a variable defined outside a function without the 'global' statement.

Example

>>> def fun():

...         global x

...         print 'x is',x

...         x=5

...         print 'changed global x to',x

Output

>>> x=10

>>> fun()

x is 10

changed local x to 5

>>> print 'Value of x is :',x

Value of x is : 5

Default Argument values

     This type of arguments are used to assign default values to the parameters.

Example

>>> def name(message,num=1):

... print message*num

Output

>>> name('Alice')

Alice

>>> name('Alice',6)

AliceAliceAliceAliceAliceAlice

    An important thing is that, default values are assigned only to the parameters which are at the end of the parameter list. That is name(message,num=5) is valid, but name(message=5,num) is not valid.

Keyword Arguments

     In some situations, functions have many parameters, but we want to only specify the values of some of them. In such situation we assign values to this parameters by naming them, this is called keyword arguments. We use the names instead of position to specify the arguments to functions.

Example

>>> def fun(x,y=2,z=10):

...       print 'x is:',x,' y is:',y,' z is:',z

Output

>>> fun(3,7)

x is: 3 y is: 7 z is: 10

>>> fun(25,z=24)

x is: 25 y is: 2 z is: 24

>>> fun(z=50,x=100)

x is: 100 y is: 2 z is: 50

    Two advantages of using the Keyword arguments are: (1) We dont worry about the order of the arguments. (2) We can give values to only those parameters that we want.

    In this example first, the parameter x gets the value 3, y gets the value 7 and the parameter z uses the default value 10. In second case, parameter x gets the value 25, y uses the default value and z gets the value 24. In third case x gets the value 100, y gets the default value and z gets the value 50. Here order of arguments are not important. We can pass argument values to parameters in any order by using the name of parameters.

Return Statement

Return statement used to return from the function. Optionally we can return value from a function by using the return statement.

Example

>>> def largest(a,b):

...         if a>b:

...             return a

...         else:

...             return b

Output

>>> print largest(67,45)

67

Every function implicitly contain an return statement. If you not return any value, that return statement return a none value.

DocStrings

Python has a feature called Documentation String, which is usually referred to as docstrings. It helps to document the program better and makes it more easy to understand.

Example

>>> def max(x,y):

...         '''Prints the maximum of two numbers.

...         the two values must be integers.'''

...         x=int(x)

...         y=int(y)

...         if(x>y):

...             print x,'is maximum'

...         else:

...             print y,'is maximum'

Output

>>> max(5,8)

8 is maximum

>>> print max.__doc__

Prints the maximum of two numbers.

the two values must be integers.

     docstring is the first logical line of a function. Note that docstring also applied to modules and classes. Here the first line start with a capital letter and ends with a dot. Then the second line is blank followed by any detailed explanation starting from the third line.

     These are the basic concepts of function in python. Detailed description of function are described in the following posts.