Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Thursday, 3 May 2018

Shell Scripting Examples

Generate comma separated dates for last seven days:

# Fetch the date 7 days ago
DATE=$(date -d '7 day ago' '+%Y-%m-%d');

# Generate comma-separated string for last 7 days
for i in {6..1}; do DATE=$DATE","$(date -d "$i day ago" '+%Y-%m-%d'); done

Considering current date is 2016-04-08, DATE variable value will be as follows:

2016-04-01,2016-04-02,2016-04-03,2016-04-04,2016-04-05,2016-04-06,2016-04-07

We can use different date formats on a case to case basis. This thing comes in handy when we have to feed a date range to some reporting tool.

Generate comma separated dates for last month and from today to start of the day:





Monday, 16 November 2015

Grep/Search specific line number in linux/unix.

Today I had a situation where I had to search/grep for a specific line number in a file in linux. Here is the solution I found and thought it is worth sharing:

Uncompressed File:
cat test_file.csv | head -line_number | tail -1
cat test_file.csv | head -337930 | tail -1

Compressed File:
zcat test_file.gz | head -line_number | tail -1
zcat test_file.gz | head -337930 | tail -1

Above examples grep/search for line number 337930.

Saturday, 17 October 2015

Imitating domain names in linux.

Sometimes we may have to simulate different domains in our local machine. For example If you want to use domain "awesome.com" in place of localhost you can do the following:

You must be having below entry in your /etc/hosts file:

  127.0.0.1 localhost

Change this entry as given below:

  127.0.0.1 awesome.com

And you are done. Hit "awesome.com" from your browser and the request will come to your localhost. Similarly you can do multiple entries in /etc/hosts to simulate multiple domains.

Wednesday, 29 October 2014

Google Protobuf - Get it working on Linux


Note: This article is posted while keeping in mind that the reader has already gone through the Proto file syntax i.e. what variable are used data types etc, last but not the least what protobuf is? If answer to this question is NO, Please visit: https://developers.google.com/protocol-buffers/

Getting Goggle Protobuf working is quite a heck so to make it easy I thought it would be worth sharing a quick read:

Steps to configure a sample maven java project in eclipse using protobuf in LINUX:
1. Install the protobuf compiler as below:
sudo apt-get install protobuf-compiler

2. Maven dependency that needs to be inserted into the pom.xml
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.6.0</version>
</dependency>

3. First of all we have to create a .proto file and then compile it using the protobuf compiler which results in .java class:
Syntax:  
protoc --java_out=<path where you want your respective java files to be created> <the actual .proto file's path>
Example: 
protoc --java_out=src/com/piyush/demo src/com/piyush/demo/addressbook.proto
protoc --java_out=src/ src/com/piyush/demo/my_object.proto

4. Earlier step can be automated using "maven-antrun-plugin", however get this running is quite a heck so please refer the below thread for this:
Links:
We can use maven-antrun-plugin in following ways:
  • For Single proto file compilation:
<echo>Generate</echo>
<exec executable="protoc">
<arg value="--java_out=src/" />
<arg value="src/com/piyush/demo/addressbook.proto" />
</exec>
  • For Multiple proto files compilation:
        <tasks>
                <echo>Generate</echo>
                    <path id="proto.path">
                        <fileset dir="src/com/piyush/demo/proto">
                            <include name="**/*.proto" />
                            </fileset>
                       </path>
                        <pathconvert pathsep=" " property="proto.files" refid="proto.path" />
                        <exec executable="protoc" failonerror="true">
                            <arg value="--java_out=src/" />
                            <arg value="-I${project.basedir}/src/com/piyush/demo/proto" />
                            <arg line="${proto.files}" />
                        </exec>
        </tasks>


Tips & Tricks:
1. Message name should be in camel case.

2. Beware that the name of the message and java_outer_classname should not be same in the proto file. For Example below proto file won't compile:
option java_package = "com.xyz.zrtb.simulator.protos";
option java_outer_classname = "A";
message a {
   optional string id = 1;
   optional string name = 2;
   repeated string cat = 4;
   optional string domain = 3;
}
Below will compile fine:
option java_package = "com.xyz.zrtb.simulator.protos";
option java_outer_classname = "OuterA";
message A {
   optional string id = 1;
   optional string name = 2;
   repeated string cat = 4;
   optional string domain = 3;
}


3. Importing proto file in one another is quite a heck and so it is good if you get it right at the very first place:
Import relies on the on the parameter '--proto_path' so If you've a package com.abc.pqr.model and defined '--proto_path' as below:
--proto_path=${project.basedir}/src/main/java/com/abc/pqr/model
THEN you can import just by writing xxx.proto However If you've defined '--proto_path' as below:
--proto_path=${project.basedir}/
THEN you can import by writing src/main/java/com/abc/pqr/model/xxx.proto and many such combinations.

4. The maven-antrun-plugin is problematic and can be a blocker, so below is the ready to use pom.xml configuration:
<build>
     <pluginManagement>
         <plugins>
            <!-- force a dependency on JDK 5.0 -->
            <plugin>
               <groupId>org.apache.maven.plugins</groupId>
               <artifactId>maven-compiler-plugin</artifactId>
               <configuration>
                  <source>1.7</source>
                  <target>1.7</target>
               </configuration>
            </plugin>
            <!--This plugin's configuration is used to store Eclipse m2e settings only. It has no influence on the Maven build
          itself. -->
            <plugin>
               <groupId>org.eclipse.m2e</groupId>
               <artifactId>lifecycle-mapping</artifactId>
               <version>1.0.0</version>
               <configuration>
                  <lifecycleMappingMetadata>
                     <pluginExecutions>
                        <pluginExecution>
                           <pluginExecutionFilter>
                              <groupId>org.apache.maven.plugins</groupId>
                              <artifactId>maven-antrun-plugin</artifactId>
                              <versionRange>[1.7,)</versionRange>
                              <goals>
                                 <goal>run</goal>
                              </goals>
                           </pluginExecutionFilter>
                           <action>
                              <ignore />
                           </action>
                        </pluginExecution>
                     </pluginExecutions>
                  </lifecycleMappingMetadata>
               </configuration>
            </plugin>
         </plugins>
        <pluginManagement>
      <plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-antrun-plugin</artifactId>
            <version>1.7</version>
            <executions>
               <execution>
                  <id>generate-sources</id>
                  <phase>generate-sources</phase>
                  <configuration>
                     <target>
                        <echo>Generate</echo>
                        <path id="proto.path">
                           <fileset dir="src/main/java/com/piyush/zrtb/simulator/proto">
                              <include name="**/*.proto" />
                           </fileset>
                        </path>
                        <pathconvert pathsep=" " property="proto.files" refid="proto.path" />
                        <exec executable="protoc" failonerror="true">
                           <arg value="--java_out=src/main/java" />
                           <arg value="-I${project.basedir}/src/main/java/" />
                           <arg line="${proto.files}" />
                        </exec>
                     </target>
                     <sourceRoot>src/main/java</sourceRoot>
                  </configuration>
                  <goals>
                     <goal>run</goal>
                  </goals>
               </execution>
            </executions>
         </plugin>
      </plugins>
      <extensions />
   </build>

Links to go through:
  • https://code.google.com/p/protoclipse/
  • http://www.siafoo.net/user/stou/blog/2010/01/29/Protocol-Buffers-and-Eclipse
  • http://www.masterzen.fr/2011/12/25/protobuf-maven-m2e-and-eclipse-are-on-a-boat/
  • http://techtraits.com/noproto/
  • http://xmeblog.blogspot.in/2013/12/sending-protobuf-serialized-data-using.html
  • http://sleeplessinslc.blogspot.in/2010/03/restful-representation-with-google.html
  • http://tutorials.jenkov.com/maven/maven-tutorial.html
  • https://developers.google.com/protocol-buffers/docs/proto#generating