Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HADOOP-19139.No GetPathStatus for opening AbfsInputStream #6699

Open
wants to merge 76 commits into
base: trunk
Choose a base branch
from
Open

HADOOP-19139.No GetPathStatus for opening AbfsInputStream #6699

wants to merge 76 commits into from

Commits on Mar 28, 2024

  1. have a way where in the first read call set the pathStatus infomation… 

    … in inputStream
    @saxenapranav
    saxenapranav committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    e2a4e05 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2024

  1. successfulUsage as atomicInteger; readFully; readOptimization conditions

    @saxenapranav
    saxenapranav committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    8d4756b View commit details
    Browse the repository at this point in the history

  2. done with src code

    @saxenapranav
    saxenapranav committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    4aae0c8 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2024

  1. readFully optimization condition change

    @saxenapranav
    saxenapranav committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    2d6f0cb View commit details
    Browse the repository at this point in the history

  2. successfulUsage

    @saxenapranav
    saxenapranav committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    5939e60 View commit details
    Browse the repository at this point in the history

  3. read buffer and length set for optimized read in case of no gps info

    @saxenapranav
    saxenapranav committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    c8a6a00 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2024

  1. read contentlen from content-range in read response

    @saxenapranav
    saxenapranav committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    4ad349c View commit details
    Browse the repository at this point in the history

  2. correct calc

    @saxenapranav
    saxenapranav committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    ee1f4c3 View commit details
    Browse the repository at this point in the history

  3. getContentLength

    @saxenapranav
    saxenapranav committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    cfc4f8e View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. testExceptionInOptimization

    @saxenapranav
    saxenapranav committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    cc09403 View commit details
    Browse the repository at this point in the history

  2. config for head optimization in abfsinputstream, config for allowing … 

    …prefetch trigger on the first read call
    @saxenapranav
    saxenapranav committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    6cb1773 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'trunk' into saxenapranav/noGpsForRead

    @saxenapranav
    saxenapranav committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    bd920a5 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. fix in contract test if head optimization is switched on

    @saxenapranav
    saxenapranav committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    37bc691 View commit details
    Browse the repository at this point in the history

  2. prefetch on first would also depened if Etag info is there; test smal… 

    …l getFs refactors
    @saxenapranav
    saxenapranav committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    7ecabf9 View commit details
    Browse the repository at this point in the history

  3. breaking readOptimized loop; getContentLength()

    @saxenapranav
    saxenapranav committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    2c9da9e View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. importnat change in abfsinputStream; tesch changes wip

    @saxenapranav
    saxenapranav committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    58a399c View commit details
    Browse the repository at this point in the history

  2. fixed test in Footer testclass

    @saxenapranav
    saxenapranav committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    7d22917 View commit details
    Browse the repository at this point in the history

  3. small read test fixed

    @saxenapranav
    saxenapranav committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    52e19c0 View commit details
    Browse the repository at this point in the history

  4. readFooter test refactor

    @saxenapranav
    saxenapranav committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    8c76674 View commit details
    Browse the repository at this point in the history

  5. fixed failing tests; footer optimization can start on fcursor ==0 as … 

    …well
    @saxenapranav
    saxenapranav committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    a06a2a1 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2024

  1. fix failing tests

    @saxenapranav
    saxenapranav committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    8f44c96 View commit details
    Browse the repository at this point in the history

  2. AbfsRestOperationException now contains the ref to abfsHttpOperation … 

    …which failed; test fixes
    @saxenapranav
    saxenapranav committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    b66eea6 View commit details
    Browse the repository at this point in the history

  3. test fix wip, when readOptimizad fail with FileNotFound, dont do read… 

    …Block
    @saxenapranav
    saxenapranav committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    45a0796 View commit details
    Browse the repository at this point in the history

  4. isNonRetriableOptimizedReadException

    @saxenapranav
    saxenapranav committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    b1fd443 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2024

  1. switch off DEFAULT_HEAD_OPTIMIZATION_INPUT_STREAM

    @saxenapranav
    saxenapranav committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    38f5592 View commit details
    Browse the repository at this point in the history

  2. switching off DEFAULT_PREFETCH_READAHEAD_ON_FIRST_READ

    @saxenapranav
    saxenapranav committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    074fb39 View commit details
    Browse the repository at this point in the history

  3. giving sync access to fileStatusInformationPresent. for un-sync metho… 

    …d accesses
    @saxenapranav
    saxenapranav committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    99e397c View commit details
    Browse the repository at this point in the history

  4. spotbugs issue

    @saxenapranav
    saxenapranav committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    fe1df56 View commit details
    Browse the repository at this point in the history

  5. Fixed readahead contract seek test for the prefetch config

    @saxenapranav
    saxenapranav committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    0475b3e View commit details
    Browse the repository at this point in the history

  6. added test for small read with the head optimization changes

    @saxenapranav
    saxenapranav committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    a1c56c2 View commit details
    Browse the repository at this point in the history

  7. added test for readFooter if it can read invalid data.

    @saxenapranav
    saxenapranav committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    dc898c8 View commit details
    Browse the repository at this point in the history

  8. fix the failing test

    @saxenapranav
    saxenapranav committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    9e7bb6c View commit details
    Browse the repository at this point in the history

  9. remove non-required assume

    @saxenapranav
    saxenapranav committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    cb929f9 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2024

  1. spotbugs; checkstyle

    @saxenapranav
    saxenapranav committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    e7b121a View commit details
    Browse the repository at this point in the history

  2. default configs done to what is there on OSS trunk; correction of SAS… 

    … test on open.
    @saxenapranav
    saxenapranav committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    51373f0 View commit details
    Browse the repository at this point in the history

  3. added comments on the metric calculations for better code readablity

    @saxenapranav
    saxenapranav committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    9b998d4 View commit details
    Browse the repository at this point in the history

  4. update in filesystem.md that fs.open might be lazy open and nonexiste… 

    …nce issue might not surface until read is done
    @saxenapranav
    saxenapranav committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    7d1a274 View commit details
    Browse the repository at this point in the history

  5. directory check and not found file check in inputStream

    @saxenapranav
    saxenapranav committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    cd9b0de View commit details
    Browse the repository at this point in the history

  6. eol fix

    @saxenapranav
    saxenapranav committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    0c454d7 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Merge branch 'trunk' into saxenapranav/noGpsForRead

    @saxenapranav
    saxenapranav committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    5c3553b View commit details
    Browse the repository at this point in the history

  2. no atomicBoolean required

    @saxenapranav
    saxenapranav committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    b224fa2 View commit details
    Browse the repository at this point in the history

  3. etag condition in header add; check if fileStatus provided which chec… 

    …king if path is directory
    @saxenapranav
    saxenapranav committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    2fc5bbd View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. remove same code method from the 3 contract class; has been braught i… 

    …n AbfsFileSystemContract
    @saxenapranav
    saxenapranav committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    e7cfec5 View commit details
    Browse the repository at this point in the history

  2. try catch for fs.open mock in AbfsFileSystemContract

    @saxenapranav
    saxenapranav committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    2f51126 View commit details
    Browse the repository at this point in the history

  3. review comments

    @saxenapranav
    saxenapranav committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    02b3421 View commit details
    Browse the repository at this point in the history

  4. review comments

    @saxenapranav
    saxenapranav committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    4c08320 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. review comments; exception catching only on FileNotFoundException

    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    f62f868 View commit details
    Browse the repository at this point in the history

  2. review comments

    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    36ba7c0 View commit details
    Browse the repository at this point in the history

  3. reduce git diff

    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    7978c27 View commit details
    Browse the repository at this point in the history

  4. added comments for better understanding

    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    8d38aed View commit details
    Browse the repository at this point in the history

  5. arraycopy conditioning for optimizedRead; added test for it

    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    811693d View commit details
    Browse the repository at this point in the history

  6. changes in durationtracker reverted

    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    065919a View commit details
    Browse the repository at this point in the history

  7. compile issue

    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    9e73700 View commit details
    Browse the repository at this point in the history

  8. Revert "changes in durationtracker reverted" 

    This reverts commit 065919a.
    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    1b3ba1b View commit details
    Browse the repository at this point in the history

  9. made ITestAbfsDurationTrackers run sequential; assertion revert revert

    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    f4fef33 View commit details
    Browse the repository at this point in the history

  10. Revert "made ITestAbfsDurationTrackers run sequential; assertion reve… 

    …rt revert"

This reverts commit f4fef33.
    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    caf6c56 View commit details
    Browse the repository at this point in the history

  11. Revert "Revert "changes in durationtracker reverted"" 

    This reverts commit 1b3ba1b.
    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    480a705 View commit details
    Browse the repository at this point in the history

  12. greater than equal to comparison

    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    e7cd9a3 View commit details
    Browse the repository at this point in the history

  13. checkstyle

    @saxenapranav
    saxenapranav committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    4154485 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. new configs set to the default as in trunk

    @saxenapranav
    saxenapranav committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    caa0756 View commit details
    Browse the repository at this point in the history

  2. javadocs; comments refactor

    @saxenapranav
    saxenapranav committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    5ddf14a View commit details
    Browse the repository at this point in the history

  3. corrected getter method name in abfsConfiguration for new configs

    @saxenapranav
    saxenapranav committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    0b53e1e View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. assertion change for readFullFile config; added comments

    @saxenapranav
    saxenapranav committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    bd86173 View commit details
    Browse the repository at this point in the history

  2. checkstyles

    @saxenapranav
    saxenapranav committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    44ffeb3 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. Merge branch 'trunk' into saxenapranav/noGpsForRead

    @saxenapranav
    saxenapranav committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    01239aa View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. added comment for better understanding on why throwing FileNotFound i… 

    …f got in optimizedRead
    @saxenapranav
    saxenapranav committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    ffefdb3 View commit details
    Browse the repository at this point in the history

Commits on May 6, 2024

  1. Merge branch 'trunk' into saxenapranav/noGpsForRead

    @saxenapranav
    saxenapranav committed May 6, 2024
    Configuration menu
    Copy the full SHA
    20f4f2b View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. Merge branch 'trunk' into saxenapranav/noGpsForRead

    @saxenapranav
    saxenapranav committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    0a1b23c View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. review refactor

    @saxenapranav
    saxenapranav committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    19c1d94 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. spotbugs

    @saxenapranav
    saxenapranav committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    429a518 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. abfs.md doc; refactored to simplify openFileForRead for HEAD call sce… 

    …nario
    @saxenapranav
    saxenapranav committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    d096f3d View commit details
    Browse the repository at this point in the history

  2. remove synchronized blocks and let native long contentLength as volatile

    @saxenapranav
    saxenapranav committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    9231959 View commit details
    Browse the repository at this point in the history

  3. code fix

    @saxenapranav
    saxenapranav committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    4547f53 View commit details
    Browse the repository at this point in the history

  4. code fix

    @saxenapranav
    saxenapranav committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    4df6b02 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. fix, abfsHttpOp only on success case and not from exception

    @saxenapranav
    saxenapranav committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    c8299fc View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. checkstyle

    @saxenapranav
    saxenapranav committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    6de6b88 View commit details
    Browse the repository at this point in the history