Difference between revisions of "Page history"
Line 7: | Line 7: | ||
<br> | <br> | ||
<br> | <br> | ||
− | === | + | ===January 10th, 2016=== |
+ | There are a few (less than 10) APIkeys which are heavily misusing the OpenCellID API in one or the other way. Last week this raised to a total of 100-160 requests every second, so more than 10 million unauthorized requests per day, and caused a noticeable drop of the performance of the OpenCellID servers. | ||
+ | |||
+ | On top of that we constantly receive ten thousands of requests every day without any APIkey or with a fake APIkey. | ||
+ | |||
+ | Therefore it was decided that these APIkeys and requests are blocked in our load balancers and will not get ANY feedback from the platform anymore. | ||
+ | |||
+ | ===November 13th, 2015=== | ||
The long awaited data cleanup has been completed!<br> | The long awaited data cleanup has been completed!<br> | ||
− | After taking the responsibility for OpenCellID we first have rewritten the entire software because former Ruby on Rails implementation was poor and could not handle the expected amount of users and data. This was completed in summer 2014. | + | After taking the responsibility for OpenCellID we first have rewritten the entire software because former Ruby on Rails implementation was poor and could not handle the expected amount of users and data. This was completed in summer 2014. |
− | Then we approached all issues regarding the improvement of the quality of the data. | + | |
− | As a first step we implemented a variety of filters to ensure filtering out all bad data from newly arriving measurements. This was completed early 2015. | + | Then we approached all issues regarding the improvement of the quality of the data. |
+ | As a first step we implemented a variety of filters to ensure filtering out all bad data from newly arriving measurements. This was completed early 2015. | ||
+ | |||
Then we started to clean up all existing data.<br> | Then we started to clean up all existing data.<br> | ||
In order to do so we have applied a number of filters to all 1.5 Billion records in the database and removed all records which are obviously wrong (e.g. a cell with German MCC located in Africa).<br> | In order to do so we have applied a number of filters to all 1.5 Billion records in the database and removed all records which are obviously wrong (e.g. a cell with German MCC located in Africa).<br> | ||
− | Due to the complex computations execution of this filtering took several weeks despite the fact that we are using several powerful servers with 128GB RAM, 16 processors etc. | + | Due to the complex computations execution of this filtering took several weeks despite the fact that we are using several powerful servers with 128GB RAM, 16 processors etc.<br> |
− | After completing this batch filtering process the first time back in June we found, that we missed to execute one step in the middle of the process | + | After completing this batch filtering process the first time back in June we found, that we missed to execute one intermediate step in the middle of the process so we had to start over again.<br> |
− | Our 2nd attempt worked as expected so now we have a clean and trustworthy database with measurements. | + | Our 2nd attempt worked as expected so now we have a clean and trustworthy database with measurements. |
− | Then we recreated all computations of the positions of cell IDs. This has been completed today. | + | |
+ | Then we recreated all computations of the positions of all cell IDs. This has been completed today. | ||
+ | |||
'''This means for you that in case you are working with the OpenCellID measurements_??.csv.gz files you should download ALL existing files again in order to work with the latest cell tower position tables.<br> | '''This means for you that in case you are working with the OpenCellID measurements_??.csv.gz files you should download ALL existing files again in order to work with the latest cell tower position tables.<br> | ||
Those of you using the OpenCellID API already enjoy the updated data. Nothing to be done in this regards.''' | Those of you using the OpenCellID API already enjoy the updated data. Nothing to be done in this regards.''' |
Revision as of 20:01, 10 January 2016
January 10th, 2016There are a few (less than 10) APIkeys which are heavily misusing the OpenCellID API in one or the other way. Last week this raised to a total of 100-160 requests every second, so more than 10 million unauthorized requests per day, and caused a noticeable drop of the performance of the OpenCellID servers. On top of that we constantly receive ten thousands of requests every day without any APIkey or with a fake APIkey. Therefore it was decided that these APIkeys and requests are blocked in our load balancers and will not get ANY feedback from the platform anymore. November 13th, 2015The long awaited data cleanup has been completed! Then we approached all issues regarding the improvement of the quality of the data. As a first step we implemented a variety of filters to ensure filtering out all bad data from newly arriving measurements. This was completed early 2015. Then we started to clean up all existing data. Then we recreated all computations of the positions of all cell IDs. This has been completed today. This means for you that in case you are working with the OpenCellID measurements_??.csv.gz files you should download ALL existing files again in order to work with the latest cell tower position tables. November 19th, 2014Version 3.5.0 has been launched with the following new features: Backend:
UI: Wiki: Next steps: cleanup of the existing data; we made some progress but this task is not yet finished; completion expected in Q1/2015 September 29th, 20141 Million unique cells have been reached in Germany:
September 1st, 2014Version 3.4.0 has been launched with the following new features: Backend:
UI: Wiki: August 23rd, 2014OpenCellID last night exceeded the number of 1.000.000.000 (1 Billion) measurements thanks to its great community of contributors!
August 22nd, 2014Version 3.3.9.2 has been launched with the following new features: Backend: UI: Wiki: August 1st, 2014Mozilla Location Service announced its collaboration with OpenCellID here July 29th, 2014Version 3.3.9.1 has been launched. Backend: UI: Wiki: July 28th, 2014Business Geomatics read here (German) July 20th, 2014Open Source Software Guide: read here July 19th, 2014Liberales Forum: read here July 18th, 2014Important German "Linux Magazin" publishes OpenCellID article. July 16th, 2014German press release: Android library has been released: Download PDF file July 7th, 2014Version 3.3.8 was launched;
June 23rd, 2014English press release: Android library has been released: Press release OpenCellID Android library June 20th, 2014German press release published for finding additional telematics service providers as contributors: Pressemeldung OpenCellID Telematik-Anbieter June 2nd, 2014Version 3.3.5.2 was launched; this is the fastest OpenCellID version to date.
|