Monday, 30 May 2016

Frequency of occurrence of Strings/Words in a list.

I had a problem where I have to keep a track of the frequency of occurrence of a given string in a record (row). I was thinking of doing this using a Map (Probably the best candidate here) where the key is the string and value is the frequency of occurrence.

But there was a catch here, what should we do on the first occurrence of the string as it won't be present in the Map. Below is the solution I came up with:

import java.util.HashMap;

public class MyHashMap {
       private HashMap<String, Integer> myHashMap;

       public MyHashMap(final HashMap<String, Integer> myHashMap) {
           this.myHashMap = myHashMap;
       }

       public void put(String key) {
           Integer integer = myHashMap.get(key);

           if (integer == null) {
                myHashMap.put(key, new Integer(1));
           } else {
                myHashMap.put(key, ++integer);
          }
        }

        @Override
        public String toString() {
            return "MyHashMap [myHashMap=" + myHashMap + "]";
         }

        public static void main(String[] args) {
             MyHashMap m = new MyHashMap(new HashMap<>());

             m.put("Steve");
             m.put("Steve");
             m.put("Simon");

             System.out.println(m);
       }
}

Output:

        MyHashMap [myHashMap={Simon=1, Steve=2}]

Above implementation works fine however you can see we had to do first occurrence handling in put method from our side which I wanted to avoid so I was looking at some other solution. No problem Java 8 Stream API is here and does this thing very smoothly in one liner.

Following is the Java 8 code snippet :

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class StreamsDemo {
public static void main(String[] args) {
List<String> list = new ArrayList<>();

list.add("Steve");
list.add("Steve");
list.add("Frank");
list.add("Tom");
list.add("Steve");

Stream<String> stream = list.stream();

Map<String, Long> collect = stream.collect(Collectors.groupingBy(e -> e, Collectors.counting()));

System.out.println(collect);
}
}

Output:

{Frank=1, Tom=1, Steve=3}

Above implementation which uses Java 8 Stream API does the trick as if we are executing a aggregate function COUNT(*) with GROUP BY. Cool !!

Sunday, 1 May 2016

Create executable jar with Maven

If you are using Maven and wants to create an executable jar file but not getting an exact solution (Google is providing too many results and nothing is straightforward :-)). So let me get this straight and there is a plugin for it which is maven-assembly-plugin.

Below is the pom.xml which makes use of maven-assembly-plugin to generate executable jar:


<?xml version="1.0" encoding="UTF-8"?>
<project
    xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
       http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example</groupId>
    <artifactId>test-project</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <properties>
        <targetJdk>1.7</targetJdk>
        <project.build.sourceEncoding>UTF-8
                          </project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8
                          </project.reporting.outputEncoding>
    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.5.1</version>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.4</version>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                            <mainClass>com.test.App</mainClass>
                        </manifest>
                    </archive>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies
                                                </descriptorRef>
                    </descriptorRefs>
                </configuration>
                <executions>
                    <execution>
                        <id>make-jar-with-dependencies</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            ......
            ......
        </dependency>
    </dependencies>
</project>

Now run "Maven install" and following executable jar file will be created : test-project-0.0.1-SNAPSHOT-jar-with-dependencies.jar

Monitoring java process running under jetty using JConsole

I came across a situation recently where an application running under jetty were having some performance issues which eventually resulted in high latency. After spending quite sometime troubleshooting the problem with no conclusion we had to profile the application to see how many threads are running, how much heap is getting used etc.

To profile the application we had chosen JConsole. Now at first it seemed to be a straightforward setup (After all its just a connection to a given host and port) however it took us a while. Hence sharing the steps here which should be followed to have a hassle free setup.

Step 1:

Add following parameters inside start.ini file found in jetty distribution directory/folder:

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.port=1099 # 1099 is default JMX Port.
-Djava.rmi.server.hostname=Your_Machine_IP


Step 2:

Make below changes inside (Adding a connector for remote JMX connection) etc/jetty-jmx.xml as per https://wiki.eclipse.org/Jetty/Tutorial/JMX:

<new class="org.eclipse.jetty.jmx.ConnectorServer" id="ConnectorServer">
<arg>
<new class="javax.management.remote.JMXServiceURL">
<arg type="java.lang.String">rmi</arg>
<arg type="java.lang.String">
<arg type="java.lang.Integer"><systemproperty default="1099" name="jetty.jmxrmiport"></systemproperty></arg>
<arg type="java.lang.String">/jndi/rmi://<systemproperty default="localhost" name="jetty.jmxrmihost">:<systemproperty default="1099" name="jetty.jmxrmiport">/jmxrmi</systemproperty></systemproperty></arg>
</arg></new>
</arg>
<arg>org.eclipse.jetty.jmx:name=rmiconnectorserver</arg>
<call name="start">
</call></new>


Note: Above xml configuration is already present inside etc/jetty-jmx.xml file but its commented by default. So just uncomment the xml tag.

Now we are all done. Open the JConsole application and connect to the application using below URL:

service:jmx:rmi:///jndi/rmi://Your_Machine_IP:1099/jmxrmi

Now we should be able to see the real-time application activity in terms of JConsole graphs.

Saturday, 2 January 2016

UI for Zookeeper - zk-web

I have been using Zookeeper for the configuration management of my application. There are usually various configurations stored in zookeeper and sometimes it happens that we have to check the configuration values. Using zk-cli command line utility we can accomplish this however zk-cli is a command line tool and you need to learn commands in order get the information out of it.

So I was looking for a GUI Tool which would be simple and cleaner and luckily I found one known as zk-web (https://github.com/qiuxiafei/zk-web).

All the installation instruction can be found here (https://github.com/qiuxiafei/zk-web), However while installing it I faced some issues and thought it would be worth sharing all the steps to have a hassle free installation.

Prerequisites:
  • Java ( > Java 6 update 40 )
  • Git
  • Lien

Installing git:

sudo yum install git (My machine was amazon ec2 Linux)
sudo apt-get install git (If the machine is having ubuntu Linux)

NOTE: Make sure you are running on java version higher than java 6 update 40 If not upgrade the java version.

Cloning git repository for zk-web:

git clone git://github.com/qiuxiafei/zk-web.git

Configuration File:

cd zk-web/conf/

[ec2-user@ip-xx-xxx-x-xxx conf]$ cat zk-web-conf.clj
{
:server-port 8080
:users {"admin" "hello"}
:default-node ""
}

Installing lein:

wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein
wget --no-check-certificate https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein

Move lein to ~/bin so that it will be available on PATH.

Grant execute permissions to lein:
cd ~/bin/
chmod 755 lein

To start zk-web run the below commands:
lein deps
lein run

Now the zk-web should be available at Your_IP:8080 URL.

Most of the time we want zk-web to run as a background process so that we can use it without the need to start it. In this scenario there is an option to generate a jar file using lein:

[ec2-user@ip-xx-xxx-x-xxx zk-web]$ cat project.clj
(defproject zk-web "0.1.0-SNAPSHOT"
     :description "FIXME: write this!"
     :dependencies [[org.clojure/clojure "1.4.0"]
       [noir "1.3.0-beta3"]
       [com.netflix.curator/curator-framework "1.1.16"]
       [com.netflix.curator/curator-test "1.1.16"]]
     :main zk-web.server)

Now using lein we have to build this clj(closure) file.
lein uberjar

This will build 2 jar files:
zk-web-0.1.0-SNAPSHOT.jar
zk-web-0.1.0-SNAPSHOT-standalone.jar

We have to use the second one which is a standalone jar. Below is command we can use to run this jar in background:
nohup java -jar target/zk-web-0.1.0-SNAPSHOT-standalone.jar &

As an alternative, we can put this command to /etc/rc.local file to start zk-web on machine reboot.

Monday, 7 December 2015

Writable redis slave

While working with redis, you may have a master redis running on a central server and all of your web application servers are running redis slaves. There may be some instances where you want to make these slaves writable for some testing purposes. Here is how we can accomplish this:

1. Load the data from master redis:
nohup /usr/bin/redis-server --port "slave port" --slaveof "Master Redis Machine IP" "Master Redis Port" > redis.out &

nohup /usr/bin/redis-server --port 7000 --slaveof 137.53.57.37 7000 > redis.out &

2. Kill the above redis process using kill command once the data is loaded.

3. Disconnect slave from master redis:
nohup /usr/bin/redis-server --port "slave port" --slaveof none "Master Redis Port" > redis.out &

nohup /usr/bin/redis-server --port 7000 --slaveof none 7000 > redis.out &

Now your slave is disconnected from master and you can write on it.

Monday, 16 November 2015

Grep/Search specific line number in linux/unix.

Today I had a situation where I had to search/grep for a specific line number in a file in linux. Here is the solution I found and thought it is worth sharing:

Uncompressed File:
cat test_file.csv | head -line_number | tail -1
cat test_file.csv | head -337930 | tail -1

Compressed File:
zcat test_file.gz | head -line_number | tail -1
zcat test_file.gz | head -337930 | tail -1

Above examples grep/search for line number 337930.

Tuesday, 3 November 2015

Webpage redirection

What happens if your visitors try to access a page that was recently renamed, deleted, or moved? By default they will get an error message, but you can change that. Instead of your visitors stumbling upon an error when trying to access such a page, you can set a redirection rule that will redirect them to a new page.

This page is about individual page redirection. For information on domain redirection check out our domain redirection tutorial.

NOTE: Redirection is very good for search engines. Search engines need to know when one of your files has changed locations to re-index your site properly.

301 and 302 redirects

There are two types of redirects - 301 and 302. These are actually HTTP status codes which specify the status of communication between the web browser and the web server.

The code samples below should be placed in the file you're redirecting from unless otherwise specified.

302 redirects

302 redirects are used when a page's location has been changed temporarily.

HTML

HTML 302 redirects work through the <meta> tag:

<meta http-equiv="refresh" content="10; url=http://www.site.com/newpage.html" />

In the content attribute, the number is the amount of seconds before redirecting to the new page specified by url.

Javascript

Javascript 302 redirects work through the window.location property:

<script type="text/javascript">
window.location="http://www.yahoo.com";
</script>

Set the window.location property equal to the URL you want to redirect to. Include this Javascript code in the head section of an HTML document. The redirect should occur right away.

PHP

PHP 302 redirects work through the header function:

<?php header("Location: http://www.site.com/newpage.html"); ?>

Inside the header() function, put Location: followed by the URL to redirect to.

.htaccess

Put this in your .htaccess file:

redirect 302 /oldfile.html http://www.yoursite.com/newfile.html

/oldfile.html is the location of the file you're redirecting from and http://www.yoursite.com/newfile.html is the location of the file you're redirecting to.

301 redirects

301 redirects are used when a page's location has been changed permanently.

PHP

PHP redirects are 302 redirects by default, but by putting the HTTP/1.1 301 code at the top, we create a PHP 301 redirect:

<?php
header("HTTP/1.1 301 Moved Permanently");
header("Location: http://www.site.com/newpage.html");
?>


NOTE: When you use a PHP redirect make sure that the redirect code is the only thing on the page that you are redirecting from.

.htaccess

Put this in your .htaccess file:

redirect 301 /oldfile.html http://www.yoursite.com/newfile.html

/oldfile.html is the location of the file you're redirecting from and http://www.yoursite.com/newfile.html is the location of the file you're redirecting to.