Skip to content
This repository was archived by the owner on Apr 19, 2022. It is now read-only.

Commit

Permalink
Snakebite Hadoop 2 compatible (HDP2)
Browse files Browse the repository at this point in the history
This commit only works on Hadoop 2.2.0 and only supports protocol
version 9 (and possibly up).

Change-Id: I46d2343bb5337f566964e159416253610a4a87f2
  • Loading branch information
Wouter de Bie committed Nov 1, 2013
1 parent 858a6d8 commit fc02dbb
Show file tree
Hide file tree
Showing 31 changed files with 3,362 additions and 2,451 deletions.
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ install:
- pip install argparse --use-mirrors
script: nosetests
env:
- HADOOP_DISTRO=cdh
# - HADOOP_DISTRO=cdh
- HADOOP_DISTRO=hdp
branches:
except:
Expand Down
10 changes: 8 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,20 @@ reasons. This is the opposite behaviour from the stock Hadoop client.

Snakebite requires python2 (python3 is not supported yet) and python-protobuf 2.4.1 or higher.

Snakebite has been tested mainly against Cloudera CDH4.1.3 (hadoop 2.0.0) in production. Tests pass on HortonWorks HDP 2.0.3.22-alpha
Snakebite 1.3.x has been tested mainly against Cloudera CDH4.1.3 (hadoop 2.0.0) in production. Tests pass on HortonWorks HDP 2.0.3.22-alpha

Snakebite 2.0.x has been tested on Hortonworks HDP2.0 and ONLY supports Hadoop 2.2.0 and up!

Installing
**********
The quickest way to install snakebite is to run:
The quickest way to install snakebite (1.3.x) is to run:

pip install snakebite

To install snakebite 2.x, clone this repo (2.0 branch) and run:

python setup.py

Documentation
*************
For more information and documentation can be found at http://spotify.github.io/snakebite/
Expand Down
6 changes: 6 additions & 0 deletions debian/changelog
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
snakebite (2.0.0) unstable; urgency=low

* Compatible with Hadoop 2.2.0 (HDP2). No longer supports client version < 9.

-- Wouter de Bie <[email protected]> Thu, 31 Oct 2013 10:29:12 +0000

snakebite (1.3.3) unstable; urgency=low

* Only support protocol version 7 and 8
Expand Down
8 changes: 5 additions & 3 deletions doc/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,13 @@ by using the ``HADOOP_JOBCLIENT_JAR`` environment variable.

Also, make sure the ``JAVA_HOME`` environment variable is set.

.. note:: Different Hadoop distributions use different protocol versions. Snakebite
and the tests default to version 7 (CDH 4.1.3), but if you want to test with
different protocol versions (HDP 2.0 uses version 8), set the ``HADOOP_PROTOCOL_VER``
.. note:: Different Hadoop distributions use different protocol versions. Snakebite 1.3.x
and the tests default to version 7 (CDH 4.1.3).
Snakebite 2.x **ONLY** supports Hadoop > 2.2.0 (protocol version >9, e.g. HDP2.0)! I
If you want to test with different protocol versions, set the ``HADOOP_PROTOCOL_VER``
environment variable to the apropriate version number.


.. note:: A hadoop installation is only required for testing.

TODO
Expand Down
154 changes: 0 additions & 154 deletions protobuf/ClientDatanodeProtocol.proto

This file was deleted.

Loading

0 comments on commit fc02dbb

Please sign in to comment.