Shell script to delete redis keys matching a pattern

A quick way to delete redis keys from the command line would be:

        redis-cli DEL "del_key"

If you wish to delete all keys matching a certain pattern,
        redis-cli KEYS "*del_pattern*" | xargs redis-cli DEL

A more easy shell script for it would be:

Save it as redis-delete.sh and you can just run it as:
        ./redis-delete.sh del_pattern

UPDATE:
The above code using KEYS does not scale well and so for a more performance efficient code, we can use SCAN which is a cursor-based iterator.

Continue

Modal dialog box and popups in Angular.js using ngDialog

If you are building an Angular.js application and wish to add a modal dialog box or a simple popup, then the ngDialog module provides an easy, simple solution. It provides two default css themes and we can also add our own custom themes.

Let's see how to create a simple dialog box with ngDialog.
For our example, we will have a base page with a button which when clicked will open a contact-us form as dialog box. The dialog box will have an 'OK' button which will save the changes and close the box and a 'Cancel' button which will just dismiss the box.

1. First, you need to download ngDialog.js and ngDialog.css (and the themes that you want) from https://github.com/likeastore/ngDialog

2. Create the base template - 'base.html'


3. Create the dialog box template - 'contact_us.html'

closeThisDialog() and confirm() are ngDialog's methods and they return an Angular promise object that is either rejected or resolved.

4. Angular.js code to handle the promise:

Now if you wish to add a popup message as per the response success or failure:

The plain option, if set to 'true' allows to use a plain string as template.
The API also provides a lot of other useful options and event handlers.

Continue

Lookahead in regular expressions

As I was solving a couple of problems in Checkio, I came across a new concept - lookaheads in regular expressions. At first, it was slightly confusing to understand how lookaheads are different from normal regular expressions. But they actually prove to be very useful in certain cases.

Lookaheads are zero-length assertions as they do not consume characters in the string but only assert whether a match is possible or not. Positive lookaheads are represented as (?=regex) and negative lookaheads as (?!regex).

Example: 

    Hello(?=World) - matches any "Hello" followed by "World"

  Hello(?=World) - matches any "Hello" which is not followed by "World" like "Hello Today" or even simply "Hello"

The real power of lookaheads arise when you chain two or more lookaheads together.

Consider a password validation which requires the password to have atleast one letter and one digit. To construct a regular expression for this case, you can't use something like, "(.*\d)(.*[A-Za-z]) " as that implies one or more digits followed by one or more letters. So a password like "123hello" would pass whereas "hello123" would fail the regex. The regex engine would start scanning the string for a digit and when it reaches "123", there are no more letters following it, so it would fail.

Instead, this can be easily achieved by using lookaheads,  "(?=.*\d)(?=.*[A-Za-z])". As the regex engine begins matching the first lookahead for digits, it traverses the string "hello123" from 'h' till it reaches the digit '1'. The first lookahead is now matched. But since the lookahead doesn't consume any string, the regex engine again starts from 'h' till it reaches a letter which is 'h' itself. Now both lookaheads are matched and the entire regex returns true.

For a more complex password validation, we could use:

((?=.*\d)(?=.*[A-Z])(?=.*[a-z])(?=.*[@?&$!]).{8,20})

which implies the password needs to have min 8 characters and max 20 characters. And it should have atleast one digit, one lowercase, one uppercase letter and one special character among "@,?,&,$,!".

Continue

Configure Nginx for load balancing

Recently, in our project, I was asked to configure Nginx for load balancing as our app was becoming slow, owing to high traffic. I haven't worked much with Nginx, so I was a bit apprehensive at first. As it was Diwali holidays, nobody else was there in office and I had no choice. However, on fumbling around, it turned out to be much more easier than I had thought.

Imagine that you wish to run two different instances of your application to distribute the high traffic.
With Nginx, you can enable all requests to be proxied to your main domain, while Nginx takes care of distributing the requests to different servers in a round robin fashion.

Open the Nginx configuration file located at /etc/nginx/nginx.conf and add an upstream block and modify the server block as shown below: 

http {
    upstream yourapp {
        ip_hash;
        server domain1.com:8080;
        server domain2.com:8080;
    }

    server {
        listen 80;
        location / {
            proxy_pass http://yourapp;
        }
    }
}

That's it! Make sure your app is running on both the servers on your specified ports and then restart Nginx by running:
/etc/init.d/nginx restart

The ip_hash specified in the upstream block is used for session persistence so that subsequent requests of the same client can be directed to the same server. Nginx uses the client's ip address as a hash key to determine which server should be selected for the client's requests. Nginx also offers other options like weighted load balancing, least connected load balancing and health checks.

Continue

Calculating the number of working days using Python

At times, we have a requirement to calculate the number of weekdays / working days between two dates.
Here's how you do it using Python:

But the above method involves usage of loops.
A more optimized way of doing it would be:

In our project, we faced a requirement where given a start date, we had to calculate end date as start date + x working days.
Using the above logic, we can do:

If you have a list of holidays to exclude, you could then do:

Continue

Stripping whitespaces and deleting duplicate rows in a PostgreSQL database

Two weeks back, in our project, we noticed that there were lot of values with leading and trailing white spaces in our database and also many duplicate row entries, as a result of which our program was not working correctly. I was assigned the task of cleaning up our PostgreSQL database.

There were two clean ups to be done:
1. Stripping of leading and trailing white spaces in string values.
2. Deleting duplicate rows from tables

I started learning a bit of plpgsql scripts to accomplish the above tasks. Since, its a very generic, common issue that most of us face, I thought I would document the steps.

Stripping white spaces:
For this, we need to fetch the list of all text columns in all tables in the database, loop though them and run a update query stripping the leading and trailing white spaces. Since the column names are available only while running the script, we need to write a dynamic sql query for the update process.

Below is the script for that:
(In all the below scripts, replace db_name with your database name)

If you also want to coalesce multiple whitespaces into a single whitespace in a string, you can use the regexp_replace function as shown below:

Deleting duplicate rows from tables:
The problem of duplicate rows will occur when the only primary key in the table is an autoincrementing id. So let's say we import data to the table by reading from a file. And if the file has duplicate rows in it, then, since there is no other primary key check, we would run into this issue. In such a case, to delete all those duplicate records, we would use:

DELETE FROM table1 a
USING table1 b
WHERE a.col1 = b.col1 and a.col2 = b.col2 and a.col3 = b.col3
--These are the non-unique columns
AND a.id > b.id; --(or a.id < b.id depending on whether we want to retain the first or last row among duplicates) 

But we need to run the above delete query for all the tables in the database. So, we need to construct a dynamic sql query that will take the table names and group by the columns present in the table.

Below is the script for that:

To execute the above scripts, save the script as a .sql file and run the below command:
psql -U db_username -d db_name -f filename.sql
 

Continue

How I got introduced to the world of geeks and fun programming

Programming. Geeks. Both were initially new to me when I entered college. Apart from few basic programs which I had done using C in high school (as part of academics), I really didn’t know much about programming. However, in college I took an instant liking to subjects like Data Structures and Algorithms which involved lots of problem solving. I enjoyed applying logic and reasoning to arrive at solutions rather than just memorizing and rote learning theory. But I was largely disappointed at the examination and evaluation pattern. The examination scheme tested neither the understanding of concepts nor the application of the concepts in real life.

In my later semesters, subjects like Compiler Design and Artificial Intelligence intrigued me even further. I was the topper in my class and I was satisfied. But then I met @justjkk, the class geek, who changed my whole view on academics. I began to realize that getting 93% in Compiler Design doesn’t in any way prove me as a good CS student. I am ashamed to say I had never used computers in my everyday life except during lab sessions and to send emails – till my 7^th semester. @justjkk pushed me into the magical world of computers and internet. I dumped my college textbooks and spent more quality time researching topics on the internet and gaining extra knowledge apart from textbooks. 

When I entered final year, we were given Placement training in our college. The sole aim of this seemed to be “How to get an IT software job without knowing any bit of programming and not caring a damn about it”. I was surprised at how any company could recruit such students and more surprised at why students would want an IT job if they weren’t interested in programming. Shockingly, many students seemed to care more about societal status rather than pursuing their passions.

Towards the end of 7^th semester, @justjkk introduced me to his best friend, @yuvipanda and he opened a new door for me – geeks and programming. I slowly began to see the whole different world of geeks – how they code with passion, how they really care about good programming and how they really apply engineering to arrive at solutions – compared to the  ‘work for a living and societal status’ attitude of normal people working in so-called reputable IT MNCs. It stirred my inner passion and I was inspired. I dreamt of becoming one among them someday, though I was unsure about my capabilities.

@yuvipanda gave me the encouragement and moral support and I started writing simple codes apart from my regular academics. I implemented algorithms like sieve of erastothenes, mergesort, quicksort, etc. They were mostly evaluated by @justjkk who then gave me valuable tips to improve. What a budding programmer can learn from an expert programmer at his side proves to be more valuable than learning got from any book.

It was during this time that I realized how much fun, programming was. Learning Data Structures and Algorithms in college was interesting. But implementing them using a programming language was even more interesting. It was kinda like solving logical puzzles which I had enjoyed since my childhood. I stumbled a lot initially while implementing some tough logic. But then the tougher it was, the more satisfaction you get by solving it. Sometimes, when I found a bug too hard to discover, I thought I would give up, thinking I hadn’t got the capabilities. But then my conscience used to taunt and mock at me. And after lots of patience and hard work, when the result came, it was more satisfactory.

I then got to know about IDEs. I worked with Eclipse and Netbeans and couldn’t help admiring the auto complete options, quick fixes and easy deployment features which made a programmer’s life much easier.

At the end of 7^th semester, we were asked to choose electives for the next semester. Majority of the class students chose subjects in terms of easiness and easy to clear. I wondered at their total uninterestedness and lack of thinking attitude. I prefer selecting a subject because I find it interesting, challenging and useful. But in the end, they won and we had to put up with boring theoretical subjects. Maybe that was an advantage on one side because during my 8^th semester, I rarely studied. My focus shifted more to learning new technical stuff. I joined a course in NIIT, Pvt. Ltd. for Java. And I started implementing small, small projects using the concepts I learnt in my daily class.

When I became fairly good in the concepts, I asked @justjkk to give me a real challenge in Java. And he suggested me to implement Game of Life. I loved anything involving Artificial Intelligence and so I got hooked on to Game of Life. I struggled with it initially and developed a crude model which @justjkk later refactored. When I looked at the refactored code, I was purely amazed and kept looking at it for several minutes. Who said only paintings and natural scenery can be beautiful? A properly written code is the most beautiful thing to the eye of a programmer. I admired the design and coding style deeply and vowed to follow it in future. Though, it took a bit more time than expected, I finally managed to develop the logic and CUI for Game of Life.

One day I decided to take up OCJP 6.0 exam. I thought it to be a good way to test my Java skills. I registered for the exam in the Prometric testing centre in NIIT, Pvt. Ltd. and they sent me a set of sample questions which they called as ‘dumps’. I went through them thinking it to be some sort of model questions. But I was really disappointed when I took up the exam. I discovered that all the questions were from the dumps and many students scored above 90% by just mugging up the options rather than understanding them. I then felt that the certification was totally worthless. I realized the saying, “Those who can, do, those who can’t, get certified.

I then learnt about version control and how to work with git, its features and advantages. I also came across Latex, a document markup language for typesetting documents. I really liked its cool features which made formatting much easier and output device independent. I used latex to create a sample resume.

One day, @justjkk informed me about Yahoo Open Hack to be held in July 30 in Bangalore. I was excited to hear about it. I had always been thrilled to hear about geek workshops and geek weekends which my friends used to attend. I thought it was a good chance to meet lots of like-minded people and geeks and immediately registered for it.

At this time, @yuvipanda told me about Thoughtworks where he had attended Functional Programming workshop. I looked up their recruitment procedure and came across the Sales Tax problem which I found interesting and decided to implement it.

Today I am learning some new stuff everyday and working toward my aspiration to become a geek. Life is fun with geeky programming.

Continue