worferrox.blogg.se - Geofencing mongodb python example

#Geofencing mongodb python example how to#
#Geofencing mongodb python example install#
#Geofencing mongodb python example driver#
#Geofencing mongodb python example code#

It took me some time to figure out how to run it and what the right syntax would be.

System.indexes>db.live.find() //To list the content The CLI command and the import to mongodb should look as the following example(I have XXX my app credentials)Ĭurl -k –get ‘ –header ‘Authorization: OAuth oauth_consumer_key=”XXXXXXXXXXXXXXXXXXXX”, oauth_nonce=” XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX”, oauth_signature=” XXXXXXXXXXXXXXXXXXXX”, oauth_signature_method=”HMAC-SHA1″, oauth_timestamp=”1409887179″, oauth_token=””, oauth_version=”1.0″‘ | mongoimport -d twitterdb -c out To get streaming data from twitter you should register an app at twitter and get a sample OAuth authorization for your app. I have listed only the changes I had to make to be able to get the example to work. If everything is OK up to this point you should be able to check the streaming examples provided with mongo-hadoop.

#Geofencing mongodb python example install#

Sudo apt-get install python3-matplotlib python3-numpy python3-scipy Sudo apt-get install python-matplotlib python-scipy python-numpysudo As an option for visualization purpose you could install scipy and numpysudo mongo-hadoop/streaming/language_support/python$ python test_install.pyħ. To test the installation and see if all pymongo modules have been installed correctly run:

#Geofencing mongodb python example driver#

The Python MongoDB Driver could be installed with:Ħ.

bashrc file:Įxport HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/nativeĮxport HADOOP_OPTS=”=$HADOOP_HOME/lib”Īnd have copied the following jar files in the same HDFS dir as the one where the lib files hdfs dfs -ls /hadoop/hadoop-2.4.0/lib Important!!! I have added the following lines in my.

After building, you will need to place the “core” jar and the mongo-java-driver in the lib directory of each Hadoop server. gradlew jar -Phadoop_version=2.4 //Phadoop should match your hadoop distribution to build the correct jar filesĤ.

#Geofencing mongodb python example code#

download the source code package from: ģ. Sudo apt-get install -y mongodb-org sudo service mongod startĬurrent stable release is 1.3.0.

Apache Hadoop – In my case hadoop-2.4.0.

Linux distribution – In my case Ubuntu Linux 14.04.1 LTS.

OK, lets roll our sleeves up and get ready. My goal in this post is to concentrate on the mongo-hadoop driver, pymongo library and the working example. There will be additional posts designed to cover these topics with greater details. I would not spend much time to explain how-to install MongoDB and Hadoop on Linux. The output is directed to MongoDB and could be easily accessed by Python or other programing languages and tools. In my scenario data is pulled from MongoDB and processed within Hadoop via one MapReduce job. The one I used and prepared is listed below. On Mongodb website you could find a large number of MongoDB and Hadoop use cases. To execute the MapReduce jobs I will use Python scripts. In this blog I will discuss the ability to use MongoDB as an “operational” real-time data store and Hadoop as an offline batch data processing and analysis platform.