sahara-extra/hadoop-swiftfs
Ray Zhang ef3c1ab4d5 Adds the lazy seek
Can make the query couple of 10 times faster.

In order to debug a Presto query performance issue, I observed the
seeking in Sahara-extra is expensive and sometimes even unnecessary.
The best way to avoid the overhead and unnecessary calls of seeking
is to do it only when the client really needs the data.
After this changes, the same query in Presto able to run 30 times faster.
Both Presto and S3 clients have added the similar changes too.

Change-Id: I8586af0d481fd08d48620e699467280f7b93150a
2016-10-05 13:26:54 -07:00
..
src Adds the lazy seek 2016-10-05 13:26:54 -07:00
README.rst doc improvement change, fixed spelling errors 2016-06-21 23:25:39 +02:00
checkstyle.xml Add checkstyle target 2014-09-16 14:07:21 +09:00
pom.xml Adds hadoop2 maven profile and fixes tests 2014-12-10 22:50:27 +09:00

README.rst

Sources for Swift filesystem implementation for Hadoop

These sources were originally published at https://issues.apache.org/jira/secure/attachment/12583703/HADOOP-8545-033.patch The sources were obtained by running "patch" command. All the files related to Hadoop-common were skipped during patching.

Changes were made after patching: * pom.xml was updated to use hadoop-core 1.1.2 dependency and adds hadoop2 profile * removed dependency on 2.x hadoop in code (@Override and isDirectory() -> isDir()) * removed Hadoop 2.X tests

There are no unit-tests, only integration.