OAuth2 with Spring Boot and MongoDB

OAuth 2 is an authorization framework that enables applications to obtain access. This is token based authentication/authorization mechanism.

I tried to secure APIs by implementing OAuth2 with Spring Boot and MongoDB. MongoDB stores user details access and refresh tokens. To know more about the configurations visit my Github Repository.

Advertisements

7 qualities of an awesome developer / consultant

Having worked across many teams with some awesome developers, I could jot down some of their qualities. These are not all the qualities they have, but I think having these qualities certainly make you a good developer or a software consultant.

1] Attention to detail
This is such an important skill to have in a developer because producing error-free deliverable’s build trust with clients. This shows that you are thorough and your solutions are solid. Creating a check-list before pushing is one of the ways to not forget things.

2] Resourcefulness
You are not sure what kind of problems you are going to be asked to solve. Problems may be completely new to you – or even to industry. When you are faced with this kind of tough problems, it’s this skill that’ll help you succeed. You should be able to solve problems quickly and creatively.

You should be able to approach your lead, manager, your peers and resources outside. Being creative in research efforts is very much important. Be sure not to quit early and show your team/client that you can get the job done.

3] Not being afraid to ask questions
Asking questions to clients or stakeholders is at the core of how a developer works. You should feel like you are an interrogator :). It also helps in attention to detail.

Asking questions help you understand what you are being asked to do. Asking questions is better than doing something wrong. If you have understood the requirements or problems, rephrasing or restating those can inspire confidence with your client.

4] Not being a yes-man
Making the customer happy does not mean you have to agree with whatever they ask or tell. Developers are closer to business, technology and end users. You should be able to find the right balance between business needs, user needs, and technical feasibility. Guide your customer towards the right approach providing right reasons.

5] They listen to others
Building great software is not a one man job, it’s a teamwork. Learn to value everyone’s opinion. Accept the fact that it’s okay to be wrong sometimes. Learn to use appropriate tone while asking/requesting someone to do something.

6] They do not ignore the silent ones
You might have some people in your team who lack participation in decision making. It might be because the person is introvert, have conflicts with others or the person does not feel that conversation is of any value to him/her. Try to include them in the discussions, take them along. You never know, they might have something really valuable which can lead to a breakthrough.

7] They don’t get easily influenced
You might end up with a team which has one or more people who are very vocal and will force you to agree with the things they believe. They are not always right, so do not get carried away.

Install docker CE on Ubuntu 14.04 (Trusty)

Install Docker CE on Ubuntu 14.04 (Trusty)

1] Uninstall older versions of docker if any.
sudo apt-get remove docker docker-engine docker.io

2] Update the repository
sudo apt-get update

3] Install recommended extra packages for Trusty 14.04
sudo apt-get install \
linux-image-extra-$(uname -r) \
linux-image-extra-virtual

4] Update apt-get and install packages to allow apt to use a repository over HTTPS
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
software-properties-common

5] Add docker GPG file
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

6] Verify fingerprint
sudo apt-key fingerprint 0EBFCD88

7] Use following commands to setup stable repository. You can also install builds from test and edge repositories.
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"

8] Update apt and install docker ce
sudo apt-get update
sudo apt-get install docker-ce

To test if docker has been installed correctly, execute the following command.
sudo docker run hello-world

To list all the docker local container:
docker ps -a

docker-ps

To list latest created container:
docker ps -l

To find out IP address, port and other information of Docker container use inspect command. This will print the JSON structure which contains all the information about the docker container.
docker inspect CONTAINER_ID

Spring Events

Spring Events

We will be discussing how to use events in Spring and will learn annotation-driven event listeners. In Spring, event publishing is the capability provided by ApplicationContext.

Few things to remember:
1] Event class should extend ApplicationEvent
2] Publisher class should have ApplicationEventPublisher object injected
3] Listener class should implement ApplicationListener interface

Consider a scenario where you have to trigger an email after any type of entity is created. An entity could be a user, project, organization, deal, task.
So the flow would be something like:
1] New entity has been created/saved in the database.
2] Create new entity created event which will be published by ApplicationEventPublisher.
3] ApplicationListener class will listen to the event and will create/send an email.

Suppose you are creating a Project. Project structure would like this:

Create Event class which extends ApplicationEvent. We are creating generic event class to handle multiple entities like projects, users, tasks etc.

Publish event after an entity is created.

Listen to the event and send mail after an entity is created.

Conclusion
In this tutorial we went over the basics of events in Spring – creating a simple, custom event, pushing it and then handling it in a listener. There is support for async processing as well. You learn more about Spring Events, visit this site.

Connection Pool Analysis

Why do we need connection pool?

I was working on a task where cron jobs were taking hours to complete, those were long running processes. To reduce the time of execution one way was to have worker threads to accomplish the jobs. I used ThreadPoolTaskExecutor and assigned the jobs, but came to know that after certain time application runs out of database connections. When I looked at the database configurations, found out we had DriverManagerDataSource data source which is not pooled data source. So to resolve this problem, decided to use a connection pool in the application.

The flow of database operation:

Database operation requires a connection. And the flow is like this:

1] Application asks the data source for a database connection.

2] Datasource uses the database driver to open a database connection.

3] Connection is created and TCP socket is opened.

4] Application performs the database operation.

5] Connection is closed and the socket is closed.

This involves opening and closing of connections which are expensive operations. This is extra strain on database management system to open / close connections.

So the best option is to reuse the database connections. This can reduce OS I/O overhead of opening/closing TCP connections.

Why pooling is useful?

Whenever a connection is requested, the data source will use the available connection pool to acquire a new connection. The pool will only create new connections when there are no connections available and a pool is not at maximum size. When the close() method is called on the connection, it’s not closed but returned to the connection pool. So reducing the overhead of creating/opening the new connections and making use of existing connections, connection pools are much faster.

Selection criteria:

Our criteria to decide which connection pool to go with.

1] Reliability

Performance is good, but it should not come at the cost of reliability. And we certainly think that reliability of library is important than performance. We don’t want to end up with the library which is faster than others but not reliable.

To decide the reliability of library, the following things we can consider:

  • How widely it is used
  • How is the code maintained
  • How active the library development is
  • Number of outstanding bugs in the library
  • Community of developers/users

2] Performance

The performance of the library again is really important criteria. The performance of libraries may vary depending upon how they are configured, the environment in which tests are carried out.

It is important to carry out the tests in the environment which you are selecting the library for.

3] Features

It is important to look at the features libraries provide. What are parameters they provide, how sensible are the default values of the parameters if we don’t provide them.

It’s important to look the connection strategies:

  • Do they reset connection properties when they return to the pool? E.g. auto-commit, isolation level
  • Do they support statement caching?

4] Ease of use

It’s also important to know how easy it is to configure (or to mis-configure) the connection pool using the library. It is important to have well documented/updated library.


Connection Pool Libraries considered for analysis:

1] Apache Commons DBCP2:

  • Easy to configure. Very detailed configuration page.
  • Supports prepared statement caching
  • Does not validate connections on borrow
  • Does not reset transaction isolation when returns to pool
  • Does not reset read only status of connection
  • Does not reset auto-commit status of connection
  • Does not clear SQL warnings
  • DBCP2 compiles and runs under Java 7 only (JDBC 4.1), not sure if it runs on Java 8
  • More than 60 classes in the library
  • Does not validate connections on borrow
  • The code is hosted on GitHub https://github.com/apache/commons-dbcp. The repository has 21 contributors and only 2 contributions in the year 2017. A lot of broken links on home page, java doc links are broken.
  • I logged into Apache Bugtracker (JIRA)  to check for the number of bugs, there are 11 open and reopened bugs.
  • Licensed under Apache 2.0

2] C3P0:

  • C3P0 is one of the oldest and best-known libraries. It is very well documented. However, according to some reports, it’s easy to misconfigure the pool and can result in poor performance and deadlocks.
  • Comes in package with hibernate.
  • Works well in the heavy concurrent environment.
  • Supports prepared statement pooling
  • Resets auto commit status when return to pool
  • Resets read the only status when return to pool
  • Resets isolation level when returning to pool
  • Does not test connections at getConnection()
  • Around 200 classes in the library. Huge code base compared to others.
  • The code is hosted on Github -> Repository, the library has 5 contributors with 5 contributions to the code this year, 4th May had the last check-in.
  • Issues are tracked on Github tracker, and has 41 issues listed.
  • Library is licensed under LGPL v.2.1 and EPL v.1.0

3] Tomcat JDBC:

  • Supports highly concurrent environments and multi core/CPU systems.
  • It is very well documented here.
  • Async connection retrieval. Can queue connection requests and return Future back.
  • Ability to configure custom interceptors to enhance functionality.
  • Provides multiple ways to configure pool. One is inside the Apache container, JMX and standalone by using DataSource bean.
  • Does not by default reset auto-commit and transaction levels for connections in the pool, users must configure custom interceptors to do this.
  • Does not by default test connections on getConnection()
  • Does not close abandoned open statements.
  • Pool prepared statement properties are not used.
  • The code is hosted on GitHub. It has 13 active contributors.
  • This is a very active community, one can see looking at the commits and bug list.
  • LLicensedunder Apache 2.0

4] HikariCP:

  • Very light library, around 130 kb.
  • Tests connections at getConnection()
  • Tracks and closes abandoned connections.
  • Clears connections before returning connection to client.
  • Resets auto-commit, transaction isolation, read-only status
  • Library has 58 active contributors and has very detailed configuration page with performance benchmarks.
  • Library has 33 issues listed on github
  • Licenced under Apache 2.0

Tests carried out with different pools:

The test results may vary with the configurations and environment they are conducted on.

I ran these tests on local developer environment with initial pool size of 10 and max size of 50.

1] I tried different connection pools and took some readings by calling database method 1000 times. Below is the table of results.

Pool & Readings (in ms) Reading 1 Reading 2 Reading 3
DBCP2 54242 48204 51760
C3P0 59204 60008 56360
HikariCP 49797 39998 46658
Tomcat CP 51156 49734 49734

 

2] Carried out tests by opening and closing the database connection thousand times. Below is the table of results.

Pool & Readings (in ms) Reading 1 Reading 2 Reading 3
DBCP2 455 359 209
C3P0 139 156 130
HikariCP 173 13 144
Tomcat CP 54 71 32

 

3] Took some readings on developer tool to monitor the response time of API (To fetch all organizations and their users).

  • Readings with Tomcat JDBC CP

pool_tomcat

  • Readings with DBCP2

pool_dbcp2

  • Readings with C3P0

pool_c3p0

  • Readings with HikariCP

pool_hikari2

Configurations used for performance testing:

1] Tomcat JDBC

2] DBCP2

3] C3P0

4] HikariCP

Final thoughts:

After going through all the resources, tomcat JDBC and HikariCP seem to be reliable and faster than other two connection pools. Both the communities are very active. Their codebase is smaller than other two. DBCP2 once was not active at one point of time but came out of dormancy later, not sure how reliable it is. Stars and forks on HikariCP and Tomcat repositories are more than that of other two connection pools.

Online resources:

http://www.trustiv.co.uk/2014/06/battle-connection-pools

https://blog.johnhagler.com/2016/08/13/pooling-mysql-jdbc-connections/

https://techblog.topdesk.com/coding/choosing-a-database-connection-pool/

https://vladmihalcea.com/2014/04/17/the-anatomy-of-connection-pooling/

https://www.scribd.com/document/111649850/Connection-Pool-Compare

http://brettwooldridge.github.io/HikariCP/ludicrous.html

https://github.com/brettwooldridge/HikariCP/wiki/%22My-benchmark-doesn%27t-show-a-difference.%22

https://github.com/brettwooldridge/HikariCP/wiki/Pool-Analysis

https://www.wix.engineering/single-post/how-does-hikaricp-compare-to-other-connection-pools

https://www.scribd.com/document/111649850/Connection-Pool-Compare

https://stackoverflow.com/questions/520585/connection-pooling-options-with-jdbc-dbcp-vs-c3p0

Prevent XSS security threat in spring application

What is XSS (Cross Site Scripting):

XSS (Cross-site Scripting) is one of the most common vulnerabilities with a web-application. And, it can be exploited by hackers very easily without using any sophisticated tool. Click here to know more about XSS.

How does it work?
Most web applications have forms to receive inputs from user. A form having text field can be exploited by a hacker.
A hacker can input javascript code e.g. or eval(“”) in the *userid* text field. When the form is
submitted, the server does process it & returns. This is not the scenario we want to happen.

Suppose your form looks like this (using spring tags):

The hacker may input something like ” jeevanalert(“I won.”) ”  in the text field. When the page is submitted, the page is returned with error as the user is not authenticated.

Solution:
If the input-text is changed into the html-escape-sequences during processing on server, then browser will receive
the escape-sequences. The resultant text would not be a valid java-script and browser will not execute it.

After conversion into html-escape-sequences, the above code would look like:

 
Since all instances of characters ‘<‘, ‘>’ and other non-alpha-numeric characters are converted into html-character-sequence, it is no longer a java-script code. So, it will not be executed by the browser.

Solution with Spring MVC In Spring-MVC, form-tags are used to create jsp page. Spring MVC provides multiple options
to encode the html-escape-sequences on server side.

  • At global level, it can be defined in web.xml file. This will be applicable to the entire application. All form-tags would refer
    to this definition. The sample code is shown below:
  • At page level, it is defined as a tag-declaration. The code is: Any form-tag, after the above declaration uses html-escape-sequence-encoding.
  • Third option is to define it as attribute for each form-tag. For example, a input-text can be defined as :

Depending upon the requirement, it can be implemented as global, page or tag level.

Filter Approach:
 Another solution to the XSS issue is to filter all the textfields in the form at the time of submitting the form. It needs XML entry in the web.xml file & two simple classes.

The code for the  first class named CrossScriptingFilter.java is:

The code second class named RequestWrapper.java is:

The only thing remained is the XML entry in the web.xml file:

The <url-pattern>/*</url-pattern> indicates that for every request made from browser, it will call CrossScriptingFilter  class. Which will parse all the components/elements came from the request & will replace all the javascript tags put by the hacker with empty string i.e. “”.

We have fours approaches to prevent the XSS threat.
You have to choose the approach according to your need.

Different folder colors in Ubuntu

If you need a tagging kind of system to track folders, you can change color of the folder. I love it.

A small extension in Nautilus can do this job. To install this extension, open terminal / terminator and execute following commands.

sudo add-apt-repository ppa:costales/folder-color

sudo apt-get update

sudo apt-get install folder-color

Once installed, restart machine or Nautilus.

nautilus -q

Once done, right click on any folder. You’ll see Folder Color option where you can change folder color to any of your choice and reset it too.

Change Folder Color

Enjoy!