Difference between revisions of "Infrastructure"

Servers

Software stack

Operating System

All OpenCellID servers are running with Ubuntu Linux 12.04 LTS.

Frontend

• The web frontend uses Apache web server as a proxy for serving web requests to Tomcat.

• The OpenCellID web application is running on Tomcat and is reading and writing cell measurements data to/from the MongoDB database backend.

• jQuery Mobile is responsible for providing a cross-platform user interface.

• The map is displayed using OpenStreetMap combined with Leaflet library.

Database Backend

The database backend, with a current 4.4 million cell towers and about 565 million measurements (1.1.2014), is a MongoDB database cluster with six servers:

• Three servers are serving as MongoDB configuration servers
• The other three servers are serving as database backend with one replication set spread across the three servers

Challenges and solutions

The OpenCellID community is very strong and continously provides a high number of measurements.
This immediately poses a few challenges:

• High Volume
  data arrives from many differnet sources and is rapidly growing
• Scale
  growth of data should go along with predictable, incremental costs and no downtime should be needed when adding additional server resources
• Data Processing
  analyzing and processing of rapidly growing data must be constantly efficient.

The current solutions are based on MongoDB and its features:

• Native Analytics
  using the integrated aggregation framework and Map/Reduce to calculate aggregates and analyses in place without the need of prior exporting data to other systems
• Advanced Geo Queries
  using geospatial MongoDB support to execute complex queries
• Horizontal Scaling
  sharding makes it easy to scale applications horizontally on commodity hardware for accommodating constantly increased throughput
• Reduced Total Cost of Ownership (TCO)
  as open-source storage MongoDB is a very cost-effective solution

The brain

Krzysztof Ociepa (email: [email protected]) has designed the big-data infrastructure as well as the new OpenCellID server software based on Java and MongoDB, and has also implemented most of the current features after two other developers failed to do so.

Details about the implemented software and infrastructure can be found above.

There are plans to publish the entire server software as open source for stimulating the contribution of software features of other community members of the OpenCellID project.
Most likely this is going to happen before the end of year 2014.