### What changes are proposed in this pull request?

Remove hostname from metrics key

### Why are the changes needed?

For easy aggregation on prometheus and grafana side

### Does this PR introduce any user facing changes?

Add a flag to disable this for compatibility

			pr-link: Alluxio#18121
			change-id: cid-ba6c2f9fae625747192044168fce7dc026c66b9c

besides the User-CLI.md doc, update other doc files that refer to `bin/alluxio` commands
- remove docs on path config
- remove starting/stopping job master/worker from contributor docs
			pr-link: Alluxio#18128
			change-id: cid-fc71dd493b16ef3aeeb0b1b190941c43b9af9cab

What changes are proposed in this pull request?
I have created a test and create a liststatus test for its function.

Why are the changes needed?
Please clarify why the changes are needed. For instance,

add a unit test for DoraWorkerClientServiceHandler.
Does this PR introduce any user facing changes?
No.
			pr-link: Alluxio#18059
			change-id: cid-b82706a4419700f017584f3e5579d2ef3410aeb3

Make fuse max reader concurrency configurable. The default value was 64 and it was unchangeable.
			pr-link: Alluxio#18129
			change-id: cid-9c55821622329bd1e608da2e7445e8ab591df38a

Fix typo from alluxio.max.fuse.reader.concurrency to alluxio.fuse.max.reader.concurrency
			pr-link: Alluxio#18134
			change-id: cid-434086cf6ba9e9f8d173e3417fc8518963dfa102

update usages of bin/alluxio, bin/alluxio-start.sh and bin/alluxio-stop.sh to their new counterparts

simplify section of CephFS.md and remove sections related to mounting. the ufs must be configured as the root mount via alluxio-site.properties.
			pr-link: Alluxio#18136
			change-id: cid-fa7d0eec00c8fb136680ef6d5a2c7ee78571d123

### What changes are proposed in this pull request?

Support accessing OSS through proxy by configuring alluxio properties or system properties.

### Why are the changes needed?

When accessing OSS through a proxy, the OSS client cannot recognize the proxy configuration in system property and environment variables. So it has to proactively set proxy-related configurations in the configuration.

### Does this PR introduce any user facing changes?

Please list the user-facing changes introduced by your change, including
  1. change in user-facing APIs
  2. addition or removal of property keys
  3. webui

			pr-link: Alluxio#18139
			change-id: cid-5e30dfd90747d4a1aafe9b2ff985331f05fefec6

### What changes are proposed in this pull request?

If don't set oss.proxy.host, the default value should be NULL

### Why are the changes needed?

If don't set oss.proxy.host, the default value should be NULL

### Does this PR introduce any user facing changes?

Please list the user-facing changes introduced by your change, including
  1. change in user-facing APIs
  2. addition or removal of property keys
  3. webui

			pr-link: Alluxio#18142
			change-id: cid-bce2790e583445c4ba6720d2f0a64551fb19de20

### What changes are proposed in this pull request?

In this change the path conversion logic is extracted to static utility methods for code reuse (because other classes may use the same path resolution logic). 

The method names are slightly improved, to distinguish the member methods in `DoraCacheFileSystem` (may be inherited) from the static utility methods.
			pr-link: Alluxio#18140
			change-id: cid-557fa148f6daa41f0b296132e5a2ecae6c5d6c22

			pr-link: Alluxio#18138
			change-id: cid-e9f862385bdfe6cc5e6938eb49907055449deb4a

### What changes are proposed in this pull request?
1. Move the path resolution logic from `DoraCacheFileSystem` to `PathUtils` where it makes more sense
2. Fix the alluxioPathToUfsPath resolution by handling the case where the ufs path may have no matching alluxio path, making the util method more generic
			pr-link: Alluxio#18146
			change-id: cid-ebace1efcf58e385bbf71b599e4b5a15a2199f7e

### What changes are proposed in this pull request?

Please outline the changes and how this PR fixes the issue.

### Why are the changes needed?

Better error handling when a stream is closed. Log more and don't swallow errors

### Does this PR introduce any user facing changes?

N/A
			pr-link: Alluxio#18145
			change-id: cid-b1146421ea5bc51e9a5eea5bf11ce1a5d8466912

Use jackson JSON as a standard format for reports.

Output example for `bin/alluxio info report summary`:
```
{
    "mSafeMode":false,
    "mZookeeper":false,
    "mRaftJournal":true,
    "version":"304-SNAPSHOT",
    "uptime":"0 day(s), 0 hour(s), 3 minute(s), and 7 second(s)",
    "rpcPort":19998,
    "webPort":19999,
    "masterAddress":"Ec2Cluster-masters-0:19998",
    "masterVersions":[
        {
            "version":"304-SNAPSHOT",
            "state":"PRIMARY",
            "host":"Ec2Cluster-masters-0",
            "port":19998
        }
    ],
    "started":"08-24-2023 06:43:32:856",
    "zookeeperAddress":[

    ],
    "raftJournalAddress":[
        "Ec2Cluster-masters-0:19200"
    ],
    "liveWorkers":2,
    "lostWorkers":0,
    "freeCapacity":"2048.00MB",
    "totalCapacityOnTiers":{
        "MEM":"2048.00MB"
    },
    "usedCapacityOnTiers":{
        "MEM":"0B"
    }
}
```
			pr-link: Alluxio#18047
			change-id: cid-bf6d54f47390a4d2bd84e4baac2ea2863d4638e1

### What changes are proposed in this pull request?
refactor the cli code

### Why are the changes needed?
make the code more flexible and easy for adding more functions in a cleaner way

### Does this PR introduce any user facing changes?
nope

			pr-link: Alluxio#18152
			change-id: cid-d8f937075174d913daf32387d781161096f03345

1. Use `addSuppressed` instead of creating a new exception to throw the original exception.
2. Go ahead to write data to UFS when it encountered exception during the time writing data to Alluxio Worker.

			pr-link: Alluxio#18017
			change-id: cid-9337252b71e40fced28fb1598ea88eed56c69229

### What changes are proposed in this pull request?

Improve distributed load
1. Configurable job failure criteria 
2. Configuration to determine if the load job should be restored from journal or not
3. Add an option to skip existing fully loaded file 
4. Add retry count for failed files 
5. Bug fixing

### Why are the changes needed?

To enhance the distributed load tool

### Does this PR introduce any user facing changes?

Yes. The skip-if-exists option is added to the distributed load cli.
			pr-link: Alluxio#18153
			change-id: cid-5644da1c09bd6ee48f628552f51cb570de581b93

### What changes are proposed in this pull request?

Recover the ufs uri support

### Why are the changes needed?

Ufs uri should be the first class citizen in dora

### Does this PR introduce any user facing changes?

No

			pr-link: Alluxio#18135
			change-id: cid-a35fe39bdf69879ef113fc97737a35ebf6d8b29a

Add docker doc back
			pr-link: Alluxio#18130
			change-id: cid-6038dce4ae1821f0e7ccf0e2e874bed5d312057d

### What changes are proposed in this pull request?

Add a test so we can monitor the vnode distribution is not too uneven. 

This test calculates the standard deviation over mean on the collection of virtual nodes assigned to physical nodes. It arbitrarily bounds it at 0.25, but ideally this number should get smaller over time as we improve hashing algorithm
and use better ways to assign virtual nodes to physical nodes.


### Why are the changes needed?

We may change hashing algorithm and virtual node assignment in the future, this will provide guidance and catch errors. 

### Does this PR introduce any user facing changes?
No. 

			pr-link: Alluxio#18147
			change-id: cid-152d8edc9b65ef59967d5985849feeb471a6650d

Add the following CLI tools for debugging and analyzing caching issues:
1. checkCaching. Checks if files under a path have been cached in alluxio.
2. location. Displays the list of hosts storing the specified file.
3. consistentHash. This command is for checking whether the consistent hash ring is changed or not.
			pr-link: Alluxio#18151
			change-id: cid-c89b98da70a5270070d873bdcfce1aa9b23cf083

Update the Golang side commands to be able to use this output:
1. Return either the yaml or json (default) output to the console.
2. Users can define the format they want with `--format` flag, like `bin/alluxio info report --format yaml`
3. In JSON format, print properties in a fixed, easy-to-read order
4. In YAML format, print properties alphabetically (since YAML specification regards property order non-significant)

Before:
```
{"safeMode":false,"masterVersions":[{"version":"304-SNAPSHOT","host":"localhost","port":19998,"state":"PRIMARY"}],"masterAddress":"localhost:19998","zookeeperAddress":[],"useZookeeper":false,"raftJournalAddress":["localhost:19200"],"useRaftJournal":true,"liveWorkers":1,"lostWorkers":0,"freeCapacity":"1024.00MB","totalCapacityOnTiers":{"MEM":"1024.00MB"},"usedCapacityOnTiers":{"MEM":"0B"},"version":"304-SNAPSHOT","webPort":19999,"started":"09-15-2023 15:54:56:635","uptime":"0 day(s), 0 hour(s), 26 minute(s), and 37 second(s)","rpcPort":19998}
```

After (in JSON):
```
{
    "rpcPort": 19998,
    "started": "09-15-2023 15:54:56:635",
    "uptime": "0 day(s), 0 hour(s), 55 minute(s), and 31 second(s)",
    "safeMode": false,
    "version": "304-SNAPSHOT",
    "webPort": 19999,
    "masterVersions": [
        {
            "version": "304-SNAPSHOT",
            "host": "localhost",
            "port": 19998,
            "state": "PRIMARY"
        }
    ],
    "masterAddress": "localhost:19998",
    "zookeeperAddress": [],
    "useZookeeper": false,
    "raftJournalAddress": [
        "localhost:19200"
    ],
    "useRaftJournal": true,
    "liveWorkers": 1,
    "lostWorkers": 0,
    "freeCapacity": "1024.00MB",
    "totalCapacityOnTiers": {
        "MEM": "1024.00MB"
    },
    "usedCapacityOnTiers": {
        "MEM": "0B"
    }
}
```

After (in YAML):
```
freeCapacity: 1024.00MB
liveWorkers: 1
lostWorkers: 0
masterAddress: localhost:19998
masterVersions:
    - host: localhost
      port: 19998
      state: PRIMARY
      version: 304-SNAPSHOT
raftJournalAddress:
    - localhost:19200
rpcPort: 19998
safeMode: false
started: 09-15-2023 15:54:56:635
totalCapacityOnTiers:
    MEM: 1024.00MB
uptime: 0 day(s), 1 hour(s), 1 minute(s), and 36 second(s)
useRaftJournal: true
useZookeeper: false
usedCapacityOnTiers:
    MEM: 0B
version: 304-SNAPSHOT
webPort: 19999
zookeeperAddress: []
```
			pr-link: Alluxio#18159
			change-id: cid-deb6e74552de9afcf45391c6c230a9fe00785e37

### What changes are proposed in this pull request?

Add datePredicate, i.e.:
lastModifiedDate(2000/01/01, 2023/09/01)

### Why are the changes needed?

Customer requirement.

### Does this PR introduce any user facing changes?

na

			pr-link: Alluxio#18155
			change-id: cid-7e1a7b7d208747807b87502c9da854ddf0b8c7fc

For random reads, bytes read per file is not a constant any more. In spite of existing duration distribution, need a throughput distribution for better understanding of reading performance.

Also, when duration too long, grpc will receive huge size of output data. Should aggregate data points to transfer more datapoints with limited output size.

Group datapoints by threads and time slices:
Example:
```
nodeResults: {
  worker-0: {
    dataPoints: [
      data: [
        { // worker 0, thread 0, slice 0
          count: 1,
          iobytes: 33554432,
        }, { // worker 0, thread 0, other slices
          …
        }
      ], [ // worker 0, other workers
        …
      ]
    ]
    throughputPercentiles: […]
  },
  worker-1: { // other workers
    …
  }
}
```

Slice time with `--slice-size` flag, e.g. `--slice-size 1s`.
			pr-link: Alluxio#18149
			change-id: cid-ec8ed5a4f9eeaa86b1d86b6b449db4647d584823

### What changes are proposed in this pull request?

Add Unit Test for OSS, OBS and GCS

### Why are the changes needed?

Unit test is important for improving functions of Alluxio.

### Does this PR introduce any user facing changes?

No

			pr-link: Alluxio#17985
			change-id: cid-c757b8249c62e2ccf0483cb99436e33d351358a1

### What changes are proposed in this pull request?

Fix datePredicate so it would respect the interval specified in the policy.
Polish tests
### Why are the changes needed?

bug fix

### Does this PR introduce any user facing changes?
na

			pr-link: Alluxio#18167
			change-id: cid-b035b5cc31aa70a2f83de2c3f84ba49ed75f9fb5

- Add a new CLI command that iterates through the command tree and generates a markdown file based on each command's definition and description
- Migrate all the content in the previous User-CLI.md into the corresponding commands in golang code, mainly updating their `Long` and `Examples` fields
- Run `bin/alluxio generate user-cli` to write the generated content directly into `docs/en/operation/User-CLI.md`
			pr-link: Alluxio#18144
			change-id: cid-9b29f273efef9693e1b0b303c62cc19602d77acc

### What changes are proposed in this pull request?

Fix the broken PagedDoraWorkerTest.

### Why are the changes needed?

The old test is.broken, I just fix it.

### Does this PR introduce any user facing changes?

no.

			pr-link: Alluxio#18150
			change-id: cid-a142297c4f08780189e4321abc8e99ff512091ec

### What changes are proposed in this pull request?

Fix UFS not loaded in some cases.

### Why are the changes needed?

Fix the following error when loading `UnderFileSystemFactory`:

```
java.lang.NoClassDefFoundError: alluxio/underfs/UnderFileSystemFactory
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at alluxio.extensions.ExtensionsClassLoader.findClass(ExtensionsClassLoader.java:73)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at alluxio.extensions.ExtensionsClassLoader.loadClass(ExtensionsClassLoader.java:82)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
	at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
	at alluxio.extensions.ExtensionFactoryRegistry.scan(ExtensionFactoryRegistry.java:248)
	at alluxio.extensions.ExtensionFactoryRegistry.scanLibs(ExtensionFactoryRegistry.java:229)
	at alluxio.extensions.ExtensionFactoryRegistry.findAllWithRecorder(ExtensionFactoryRegistry.java:149)
	at alluxio.underfs.UnderFileSystemFactoryRegistry.findAllWithRecorder(UnderFileSystemFactoryRegistry.java:115)
	at alluxio.underfs.UnderFileSystem$Factory.createWithRecorder(UnderFileSystem.java:111)
	at alluxio.underfs.UnderFileSystem$Factory.create(UnderFileSystem.java:93)
```

This was because when `Thread.currentThread().getContextClassLoader()` is null, it indicates that the class loader is actually the system class loader. If using `null` for the default class loader, it will not be able to resolve `UnderFileSystemFactory` which was already loaded by the system class loader.

### Does this PR introduce any user facing changes?
No.
			pr-link: Alluxio#18169
			change-id: cid-d79281b3fb14e894e4ff21fe34222ef9e04f05d4

Alluxio does not have a valid Block Info at this moment.

### What changes are proposed in this pull request?

return an all zero block info for fsstat

### Why are the changes needed?

No valid block info is managed by Alluxio at this moment.
Alluxio does not have 'master' node, and no block info is stored/managed by other components.

This may change in future, e.g. statfs() can gather the block info from all worker nodes.

### Does this PR introduce any user facing changes?

N/A

			pr-link: Alluxio#18172
			change-id: cid-eda9fb4ad68fa91d6d766285f6de211d5f02df56

			pr-link: Alluxio#18174
			change-id: cid-996586c993d3920cd91c4663a8771ced840fe80f

			pr-link: Alluxio#18177
			change-id: cid-9e895702ab3f800a47fed97ce2c1a756abb47819

The configuration `alluxio.user.consistent.hash.virtual.node.count` is for total virtual nodes of all workers. We need to adjust the configuration according to the number of worker nodes. However, it is difficult to determine how many virtual nodes are enough to avoid the data skew issue.

Based on this background, a better way to solve the data skew issue is to allow user to set the virtual nodes number for a worker node, but not the total virtual nodes of all worker nodes. This is what this PR does.
			pr-link: Alluxio#18179
			change-id: cid-fbb4c9ebe13f5c0cce4ac08f9fe1789d97cdf209

### What changes are proposed in this pull request?

Fix GCSV2UnderFileSystem initialization

### Why are the changes needed?

not able to create ufs instance if not set access key, which should not be a requirement for gcs v2

### Does this PR introduce any user facing changes?

na

			pr-link: Alluxio#18184
			change-id: cid-c736612c8ccc2d5d8bd9ecbd9f862942625e8156

And refactor fuse start code into startCommon() and stopCommon()

### What changes are proposed in this pull request?

Stop Fuse web server when exit.

### Why are the changes needed?

Fuse web server must be stopped, otherwise the main thread is waiting for it forever.

### Does this PR introduce any user facing changes?

N/A

			pr-link: Alluxio#18190
			change-id: cid-cdad62f25870a6a1e1984127834402045b9763d4

### What changes are proposed in this pull request?

Only stop processes belong to the current user while running `./bin/alluxio process start` or `./bin/alluxio process stop`

### Why are the changes needed?

Without this PR, it will try to stop all processes match the class name even they are not started by the current user. Like the below case

```
$ ./bin/alluxio process stop local
WARN[2023-09-21T14:57:31+08:00] No process to stop because could not find running process matching alluxio.worker.AlluxioWorker
INFO[2023-09-21T14:57:31+08:00] Found 1 running process(es) matching alluxio.master.AlluxioMaster
INFO[2023-09-21T14:57:31+08:00] Successfully killed 0 process(es)
ERRO[2023-09-21T14:57:31+08:00] Failed to kill 1 process(es):
ERRO[2023-09-21T14:57:31+08:00] Failed to kill process: error sending TERM signal to process for alluxio.master.AlluxioMaster with pid 17055
 --- at /tmp/alluxio1717887242/cli/src/alluxio.org/cli/env/process.go:175 (BaseProcess.Stop.func1) ---
Caused by: operation not permitted
```

### Does this PR introduce any user facing changes?

NO

			pr-link: Alluxio#18187
			change-id: cid-ee45c89e93fff4bd8c58dba3e81cd1fa4de52368

### What changes are proposed in this pull request?
Fixed a typo

### Why are the changes needed?

### Does this PR introduce any user facing changes?
No

			pr-link: Alluxio#18191
			change-id: cid-67a503376ed89395ebccd674644394e5bc02522b

### What changes are proposed in this pull request?
Don't call the ufs.close() in PageFileReader.close()

### Why are the changes needed?
1. we should not close the ufs instance, because we always try to reuse it for the same ufs mount point. unless we don't reuse the instance of the ufs instance.
2. S3UnderFileSystem.close() will just shut down the executor held by the instance. when other threads reuse the executor, it will cause issues.

### Does this PR introduce any user facing changes?
no

			pr-link: Alluxio#18188
			change-id: cid-2d97d2ba48bb8dc333cb8d2dddd7990dad5d372b

### What changes are proposed in this pull request?

1. Add an abstract representation of a worker's identity.
The new identity is the same as a `long` based identity that used to be generated by the master. It decouples the concept of an identity and the actual implementation of an identity. It also includes a version field that allows backward compatible evolution of the worker identifier.

2. A new implementation for worker identities that uses UUID.
A worker will generate its own identity instead of acquiring one from the master, unless the user explicitly specifies one in the configuration. The generated identity is then persisted in the worker's working directory, and reloaded at the next worker restart.

### Why are the changes needed?

For Dora workers, the identities are not generated by a centralized service, but by workers themselves. Therefore, we need something that provides randomness and avoids collision between workers, e.g. a UUID. The V1 parser implementation included in this PR uses a UUID as the worker identity.

### Does this PR introduce any user facing changes?

Yes, a new configuration item `alluxio.worker.identity.uuid` is added that allows a user to specify the worker identity.

			pr-link: Alluxio#17820
			change-id: cid-3200e78fa24be0126d4d49370f0a266f8c766e12

dev docs for generating user cli documentation
			pr-link: Alluxio#18192
			change-id: cid-ee3ad14a4a284ddf2040850f4d590e1ccb70cb69

### What changes are proposed in this pull request?

Split the fuse stream close() method into two individual methods:
1. Close the fuse stream
2. Release the lock

Currently no behavior changes on the close function as the function will call these two functions to achieve the same functionality as before. 

### Why are the changes needed?

When we handle some fuse operations, we sometimes want to close the stream only but want to keep the lock there, to avoid inconsistency.  

### Does this PR introduce any user facing changes?

Please list the user-facing changes introduced by your change, including
  1. change in user-facing APIs
  3. addition or removal of property keys
  4. webui

			pr-link: Alluxio#18180
			change-id: cid-93e1f5c6c2448536321cbcbb528471b032c4c01f

### What changes are proposed in this pull request?
Refactor some of the code in java and go to make the CLI more flexible for the future

### Why are the changes needed?
There might be more commands added in the future

### Does this PR introduce any user facing changes?
nope
			pr-link: Alluxio#18197
			change-id: cid-00c52688dd60bc0a451cfbff932b1da329549993

### What changes are proposed in this pull request?

Improve the randomness of StressWorkerBench random read test, now each thread throw dice every time it trying to do a random read.

### Why are the changes needed?

In previous StressWorkerBench random read test each thread read from same offset and same length everytime, this cause low randomness.

### Does this PR introduce any user facing changes?

no
			pr-link: Alluxio#18000
			change-id: cid-974afb58c022061e8a6d8d6894415a8bd292b764

Cherry-pick of existing commit.
orig-pr: Alluxio#18159
orig-commit: Alluxio/alluxio@86308c3
orig-commit-author: twalluxio <[email protected]>

			pr-link: Alluxio#18166
			change-id: cid-deb6e74552de9afcf45391c6c230a9fe00785e37

collectAlluxioInfo calls bin/alluxio commands and collects their output. fix the commands after the golang CLI refactor
			pr-link: Alluxio#18186
			change-id: cid-23c99f152405d04dcdd9bbea3953b00e23ee3882

### What changes are proposed in this pull request?

- made some funcs/consts public
- tarball artifacts list lib/plugin modules in `metadata` field

### Why are the changes needed?


### Does this PR introduce any user facing changes?

no
			pr-link: Alluxio#18195
			change-id: cid-7f7738ef1f2d4e9907dd5d9e047870f048cdd269

### What changes are proposed in this pull request?

Fix last modified time is 0 in UfsFileIterable

### Why are the changes needed?

bug fix

### Does this PR introduce any user facing changes?
na


			pr-link: Alluxio#18200
			change-id: cid-e3fc548f65b642f8b8cb62d3b94f26366e33e4a7

### What changes are proposed in this pull request?
  UfsFileIn/OutStream Test runnable now


### Why are the changes needed?
  Need UfsFileIn/OutStream Test

### Does this PR introduce any user facing changes?
  No



			pr-link: Alluxio#18170
			change-id: cid-2de71fa1d5bc6189c03f6fd496f55c5c53fd778c

By default, we build a dynamic consistent hash with the live worker list that comes from master or ETCD. Sometimes we want to build a static consistent hash ring to make sure we won't write data to other worker node when a worker node is offline temporarily (especially when other worker nodes are running out of disk space).

This PR provides allows us to build a static consistent hash ring by setting `alluxio.user.dynamic.consistent.hash.ring.enabled=false`. In this case, client will read from UFS if the worker where the specified file locate is down.
			pr-link: Alluxio#18183
			change-id: cid-ccd37436391d62aa7c6a9942c80e2d3c8ffdb170

### What changes are proposed in this pull request?

Add client read metrics

### Why are the changes needed?

These metrics are helpful.

### Does this PR introduce any user facing changes?

N/A

			pr-link: Alluxio#18160
			change-id: cid-08d84f39da7aec0ca21e1d57731930a67693dbeb

Support dora read to fallback to UFS if any error occur during read.

			pr-link: Alluxio#18154
			change-id: cid-755c5bdbe4edf9abf32a7d700d17617cbb8a78a1

### What changes are proposed in this pull request?

1. Since EtcdClient is a singleton, close it in the jvm shutdownHook
2. Close related resources within EtcdClient

			pr-link: Alluxio#18204
			change-id: cid-b00c40829136c88dd6a6e1c0731b1a5ee3647cff

### What changes are proposed in this pull request?

1. The length of the read is incorrectly calculated. Should not have anything to do with the position of the reader.
2. `mPos` should not be updated for the `read` method, as it's a positioned read operation (has an `offset` argument).

### Why are the changes needed?

bug fix

### Does this PR introduce any user facing changes?
No.

			pr-link: Alluxio#18205
			change-id: cid-eb56603cad6c462e799793ca1086d3638209af00

### What changes are proposed in this pull request?

This improves resource deallocation in two ways:
1. Close FileSystemContext
2. Close an optional thread pool on normal exit (before it is only destroy on error)

### Why are the changes needed?

Resource deallocation
			pr-link: Alluxio#18206
			change-id: cid-03ec1f712460172dd8d61a34fc65b95484243a3b

move code into separate package
			pr-link: Alluxio#18207
			change-id: cid-91860a65af36b70f98e8913f0710e6a295d14008

### What changes are proposed in this pull request?

Please outline the changes and how this PR fixes the issue.

### Why are the changes needed?

Please clarify why the changes are needed. For instance,
  1. If you propose a new API, clarify the use case for a new API.
  2. If you fix a bug, describe the bug.

### Does this PR introduce any user facing changes?

Please list the user-facing changes introduced by your change, including
  1. change in user-facing APIs
  2. addition or removal of property keys
  3. webui

			pr-link: Alluxio#18209
			change-id: cid-3d147d628a6be3a0e654eec338ea76d2fb5f25cf

### What changes are proposed in this pull request?

remove master host check

### Why are the changes needed?
master host should not be a hard requirement right now.

### Does this PR introduce any user facing changes?
na

			pr-link: Alluxio#18201
			change-id: cid-7cfc7a688e925137a1e9d62e4af0a07298f447d4

Add DoraMetaManagerTest.
			pr-link: Alluxio#18127
			change-id: cid-7c8f875ce957e85597c53bc5db611f1d533596ee

For now StressWorkerBench uses milliseconds as the smallest unit of recording. In random read, if file is very small, it is possible to have a duration <1ms. Since instant throughput = bytes read / duration, it will lead to a divide by zero error.
This PR replace millisecond-level record of a file read with a nanosecond-level approach. It also makes the following updates on StressWorkerBench:
- record file read time within `applyOperation()` function, instead of before entering or after exiting it
- wrap metrics about the `applyOperation()` function to a private class, instead of returning bytes read only
- remove getter and setters for WorkerBenchDataPoint, since all elements inside is public
- standardize the unit of output throughput, now unit is MB/s
			pr-link: Alluxio#18175
			change-id: cid-a606ca5812f75d8116ecaf7da45da9dedd78ab55

### What changes are proposed in this pull request?

Added a manual double-safety `System.exit()` at the end of FUSE lifecycle, to be absolutely sure we exit the JVM and terminate all non-daemon & daemon threads.

Before this change, after we `alluxio-fuse umount` or `kill (not -9)`, the FUSE process may fail to quit. One reason is a non-daemon thread dangling around:
```
# This is a non daemon thread in AlluxioEtcdClient
"vert.x-eventloop-thread-0" Alluxio#21 prio=5 os_prio=31 cpu=482.23ms elapsed=722.11s tid=0x00007fde79251800 nid=0x8c03 runnable  [0x00007000102ae000]
   java.lang.Thread.State: RUNNABLE
   at sun.nio.ch.KQueue.poll([email protected]/Native Method)
   at sun.nio.ch.KQueueSelectorImpl.doSelect([email protected]/KQueueSelectorImpl.java:122)
   at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
   - locked <0x00000007c0603938> (a io.netty.channel.nio.SelectedSelectionKeySet)
   - locked <0x00000007c06038d8> (a sun.nio.ch.KQueueSelectorImpl)
   at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:136)
   at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
   at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883)
   at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526)
   at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
   at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
   at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
   at java.lang.Thread.run([email protected]/Thread.java:829)
```

The JVM shutdownHook executes after the LAST non-daemon thread dies or when `System.exit()` triggers. So if some libraries we use or some thread pools of ours accidentally introduce non-daemon threads, `System.exit()` is our last resort to trigger shutdownHook.

FUSE does handle signals and exit, but there are some corner cases which don't seem to trigger that successfully. So this PR serves as the last resort.
https://github.com/Alluxio/alluxio/blob/527ffe8105568757e05dc81dff1e28a8078e9a9e/dora/integration/fuse/src/main/java/alluxio/fuse/FuseSignalHandler.java#L52

### Why are the changes needed?

See above

### Does this PR introduce any user facing changes?

No

			pr-link: Alluxio#18211
			change-id: cid-adeaeadfaa5fb4013a62ea8191005b93c0ca69ad

### What changes are proposed in this pull request?

Prepare project structure for native components.
Add a maven archetype for Rust-based native components.
The `native` module is behind a profile `native-components` and is deactivated by default.

### Why are the changes needed?

Add support for native components

### Does this PR introduce any user facing changes?

No.

			pr-link: Alluxio#18164
			change-id: cid-530dd723b84d69cef22b08f27eae7e4b4874995a

### What changes are proposed in this pull request?

Make copy & move return list of tasks

### Why are the changes needed?
make worker busy
### Does this PR introduce any user facing changes?
na

			pr-link: Alluxio#18215
			change-id: cid-d0c3abe26ec4cee3cdbd93e2a99ffb319b187524

### What changes are proposed in this pull request?

I have created a init test to check whether the workers has already run and connected to the leader.

### Why are the changes needed?

Because the version has updated to 3.0 but we still do not have the correct init test for the cluster.

### Does this PR introduce any user facing changes?

Yes, users may still not run test when they first run the cluster because i must communicate with leader to decide which args will be used.

			pr-link: Alluxio#18148
			change-id: cid-74a018569e1ba13c55573ba9e1713bf9dd8e3f2d

### What changes are proposed in this pull request?

Please outline the changes and how this PR fixes the issue.

### Why are the changes needed?

Please clarify why the changes are needed. For instance,
  1. If you propose a new API, clarify the use case for a new API.
  2. If you fix a bug, describe the bug.

### Does this PR introduce any user facing changes?

Please list the user-facing changes introduced by your change, including
  1. change in user-facing APIs
  2. addition or removal of property keys
  3. webui

			pr-link: Alluxio#18223
			change-id: cid-375b97fb3beb2714a75dbb227c15978e18c2909a

also update `init format` command to be able to format journal on all masters but avoid formatting workers
			pr-link: Alluxio#18124
			change-id: cid-b8fd99ec5c6e6b7dbaa1c9a22414fad2a33caf50

- downgrade viper dependency to be compatible
- remove any native golang calls introduced in 16+
- update version check for cli compilation script
- add license check skip flag in tarball build
			pr-link: Alluxio#18218
			change-id: cid-bc73e4d4a8c6c78a189f213dbb11ddd4b77e5454

### What changes are proposed in this pull request?

Allow AlluxioEtcdClient to use username/password to talk to etcd if target etcd server has authentication enabled.

### Why are the changes needed?

as above.

### Does this PR introduce any user facing changes?

N/A

			pr-link: Alluxio#18193
			change-id: cid-4d2644b1bf1d600a8ff4dcffa18cbbfa8f0f6ad0

### What changes are proposed in this pull request?

add membership module doc

### Why are the changes needed?

N/A

### Does this PR introduce any user facing changes?

N/A

			pr-link: Alluxio#18233
			change-id: cid-ffea2ab2522a5c374e373f9e1c73dce36a5f38eb

### What changes are proposed in this pull request?

Fail directly if sudo needs password.

### Why are the changes needed?

Fix Alluxio#18194

### Does this PR introduce any user facing changes?

NO

			pr-link: Alluxio#18219
			change-id: cid-ad2566d5d69cdbe77dcb35302c5268c9ead4f848

Removed an unsupported test.
Added new test for `FileOutStream.write(int)`
Fixed `AbstractFileOutIntegrationTest`

			pr-link: Alluxio#18189
			change-id: cid-a0a3c2ab1f1eee9bc24240e695082ef17badeff0

### What changes are proposed in this pull request?

Add unit test for DefaultPageMetaStore.commitFile() method, and rename some variables to more accurate names. 

### Why are the changes needed?

Improve the test coverage.

### Does this PR introduce any user facing changes?

No.
			pr-link: Alluxio#18162
			change-id: cid-e7de0d5b474d36b67c85f0ed0a49202a37dc7194

Improve list files RESTful API providing more useful info. This PR add the following info to the response JSON:
1. Formatted file size
2. Alluxio path
3. UFS path
4. Last modification timestamp
			pr-link: Alluxio#18245
			change-id: cid-b7260c0ae964b31cfe67eead92f693b40b6688fc

### What changes are proposed in this pull request?

The old UfsBaseFileSystem cannot run, this pr fix it.

### Why are the changes needed?

Improve code quality.

### Does this PR introduce any user facing changes?

no.

			pr-link: Alluxio#18163
			change-id: cid-7b19138cb9db0d8b02613d96af9e6f33bc807b5d

### What changes are proposed in this pull request?

Set PropertyKey.DORA_CLIENT_UFS_FALLBACK_ENABLED to false.

### Why are the changes needed?

As we mentioned in [here](Alluxio#18213).
When we want to test the DoraFileSystem, we have to disable the Ufs Fallback, otherwise the methods like `createFile`  `getStatus` will fall back to Ufs, while we want to test DoraFileSystem.

**This is a important change, so it is necessary to open a new PR for it.**

### Does this PR introduce any user facing changes?

No.

			pr-link: Alluxio#18244
			change-id: cid-d81c5b5298fb9e7eb98fc5edde236f6fda05e0f0

### What changes are proposed in this pull request?

The LocalAlluxioCluster used in integration tests is broken when Ufs fallback is disable, this PR is used to fix it. The old master and worker liveness detection mechanism is invalid in ALluxio 3.0 arch, this PR also fix it.

### Why are the changes needed?

Fix broken tests.

### Does this PR introduce any user facing changes?

No.

			pr-link: Alluxio#18213
			change-id: cid-6e7d3cf73c80aa45f67007f90605a78654924614

### What changes are proposed in this pull request?

when etcd lease keepalive client returned with onCompleted or onError, the lease might not directly get expired at the time, therefore we always new a lease to resume keepalive without explicitly checking isleaseexpired

### Why are the changes needed?

when etcd got induced network unstableness such as pkg loss, there's race condition in our reconnection logic from ServiceDiscoveryRecipe, where we directly check isLeaseExpired when we got terminal callback from keepalive client, but at the time the lease isn't expired yet. Hence we stop trying to create new lease to keepalive.

### Does this PR introduce any user facing changes?

No

			pr-link: Alluxio#18217
			change-id: cid-bdfc1d37b359080585ac92b5daa9fac31598ea9b

Avoid creating `FileSystem` instance many times in `HttpServerHandler`, and call the `close()` method when the handler is removed.
			pr-link: Alluxio#18248
			change-id: cid-9c4f1369a9549d58b8191214547046cd1e188d48

### What changes are proposed in this pull request?

Remove MigrateIntegrationTest and PersistIntegrationTest.

### Why are the changes needed?

MigrateIntegrationTest and PersistIntegrationTest are related to the job service, which no longer exists in the Dora architecture. Therefore, MigrateIntegrationTest and PersistIntegrationTest can be removed.

### Does this PR introduce any user facing changes?


			pr-link: Alluxio#18247
			change-id: cid-f38d5660f6b7c91b8407e0740aec04d2cccb8b89

### What changes are proposed in this pull request?

Fixed one test case in FileSystemBlockLocationIntegrationTest

### Why are the changes needed?

Logic of getBlockLocation in DoraCacheFileSystem has chagned, so related IntegrationTest need be change as well

### Does this PR introduce any user facing changes?

no
			pr-link: Alluxio#18243
			change-id: cid-39521af14dafcb91d5368a25091c1031651c6dc6

### What changes are proposed in this pull request?

Supports setXattr API of Alluxio FileSystem, it will set the corresponding attributes to UFS. If UFS is a filesystem, it's set through the setXattr interface of UFS. if UFS is object storage, it's set through setTagging API.

### Why are the changes needed?
Based on the 3.x architecture, recover support for the setXattr interface. There are still many places where the interface is used, and it should not be supported at present.


			pr-link: Alluxio#18053
			change-id: cid-4f8c96c88cced4374d54b6387bfc618b47d2a423

### What changes are proposed in this pull request?

Remove BackupCommandStateLockingIntegrationTest, ReportEmbeddedJournalIntegrationTest, and 
 MetricsCommandIntegrationTest

### Why are the changes needed?

Remove invalid tests.

### Does this PR introduce any user facing changes?

no.

			pr-link: Alluxio#18251
			change-id: cid-fe71cc37517fc3ad1eb3f5dba4b05f5503afaa5e

### What changes are proposed in this pull request?
This PR adds a `--mode` option for StressWorkerBench. This option allows user to choose from a range of file read policies. Possible option values are `hash`(default) and `local-only`.

If use `hash`, the config `alluxio.client.file.dora.ConsistentHashPolicy` is set to `alluxio.client.file.dora.ConsistentHashPolicy`.

If use `local-only`, that config is set to `alluxio.client.file.dora.LocalWorkerPolicy`.

The benchmark will parse this option, and choose the right policy. Also, it will print a log about the policy the user is using in both the benchmark and the policy factory.

### Why are the changes needed?
In previous versions, the default policy is `local-only`.

However, in Alluxio, the default policy is `hash`, using `alluxio.client.file.dora.ConsistentHashPolicy`.

If user wants to use the `local-only` policy, add `--mode local-only` to the end of the benchmark command.
			pr-link: Alluxio#18246
			change-id: cid-29676a12710b9c815f3e55f84eb5e4226fea9ad7

Fix `FileInStreamIntegrationTest`, removed unnecessary test cases
			pr-link: Alluxio#18178
			change-id: cid-dac1c4c2e4e403bbf4ad3cbdde9eaf43ce20e046

### What changes are proposed in this pull request?

Add a property key to control if alluxio write the xattr to UFS

### Why are the changes needed?
The current S3 mock doesn't support setTagging/getTagging. And in some cases, we don't want to write xAttr to UFS. So Add a switch to disable it.

### Does this PR introduce any user facing changes?

			pr-link: Alluxio#18255
			change-id: cid-ccc2971f9bbb5cdd5abf31dc9fd845a1f26097b5

### What changes are proposed in this pull request?

Fix a bug of PagedFileReader. The PagedFileReader.read() method should use mFileSize, instead of mPos.

Then add unit tests of PagedFileReader.

### Why are the changes needed?

improve code coverage

### Does this PR introduce any user facing changes?

no.

![image](https://github.com/Alluxio/alluxio/assets/30886176/49f3d75b-2b9d-49e7-a089-b19ba027c7d1)



			pr-link: Alluxio#18107
			change-id: cid-0b929b7a1a8c9e32bb53be861e41f12b64d16183

### What changes are proposed in this pull request?

Use linked hash set to keep sequence when picking multiple worker

### Why are the changes needed?
When loading multiple replica into Alluxio, we want deterministic sequence so we can decide which worker to load data and which worker the client can talk to.

### Does this PR introduce any user facing changes?

na

			pr-link: Alluxio#18235
			change-id: cid-1193f29eda77cd82af8dea4bc59914e97d1f8fa3

### What changes are proposed in this pull request?

Allow `WorkerIdentity` to be de/serialized to JSON and Java serialization format.

### Why are the changes needed?

To be able to be embedded in `WorkerInfo` and other data objects that need to be de/serialized to JSON.

### Does this PR introduce any user facing changes?

No.

			pr-link: Alluxio#18224
			change-id: cid-9bd20ac6b1934feee10d130633e27323750efd17

address Alluxio#18249

refactor how java opt env vars are handled. previously they were treating as strings and then using a split operation to separate the single string into cmd line arguments. now they are handled as `[]string` immediately after parsing from env to properly handle any leading or trailing whitespaces.
			pr-link: Alluxio#18252
			change-id: cid-44c4e14465432b0a5d9d8e84b488e9cd13c1e11b

### What changes are proposed in this pull request?

Use new worker identity for worker registration and client-side cluster management.

### Why are the changes needed?

To allow workers to use generated IDs that are not assigned by the master.
To allow clients to build its view of the cluster using more efficient and reliable worker identity implementation.

### Does this PR introduce any user facing changes?

The worker identity is now persisted in etcd, if the worker uses etcd-based registration.

			pr-link: Alluxio#18210
			change-id: cid-b044e5648329715bd59172a6fd39b406ef1cdad3

### What changes are proposed in this pull request?

Check the `-f, --load-metadata` option at the worker side.

### Why are the changes needed?

The [doc](https://docs.alluxio.io/os/user/edge/en/operation/User-CLI.html#fs-ls) has

![2023-10-03_12-03-58](https://github.com/Alluxio/alluxio/assets/7149512/e839b2e5-ef2a-4834-b421-b414353572f4)

and the "fs ls" command has the following option

```
 -f, --load-metadata       Force load metadata for immediate children in a directory
```

But this option is only set at the client side and is not checked at the worker side. So setting it has no use. Please check the following example

```
$ ./bin/alluxio fs ls /
-rw-rw-r--  test test              1366                 10-02-2023 17:39:54:284 FILE /file.01
-rw-rw-r--  test test              1366                 10-02-2023 17:41:37:496 FILE /file.02
-rw-rw-r--  test test              1366                 10-03-2023 11:04:18:177 FILE /file.03

$ ./bin/alluxio fs ls -f /
-rw-rw-r--   test test             1366                 10-02-2023 17:39:54:284 FILE /file.01
-rw-rw-r--   test test             1366                 10-02-2023 17:41:37:496 FILE /file.02
-rw-rw-r--   test test             1366                 10-03-2023 11:04:18:177 FILE /file.03

$ ./bin/alluxio fs ls -Dalluxio.user.file.metadata.sync.interval=0 /
-rw-rw-r--   test test             1366                 10-02-2023 17:39:54:284 FILE /file.01
-rw-rw-r--   test test             1366                 10-02-2023 17:41:37:496 FILE /file.02
-rw-rw-r--   test test             1366                 10-03-2023 11:04:18:177 FILE /file.03
-rw-rw-r--   test test             1366                 10-03-2023 11:09:33:642 FILE /file.04
-rw-rw-r--   test test             1366                 10-03-2023 11:12:57:668 FILE /file.05
```

This PR checks it at the worker side and makes it matches the doc.

### Does this PR introduce any user facing changes?

NO

			pr-link: Alluxio#18238
			change-id: cid-d8e77340861e2b78e8908012b4fe262421202909

### What changes are proposed in this pull request?

Remove the @ignore annotations. It works now.

### Why are the changes needed?

improve code quality.

### Does this PR introduce any user facing changes?

no.

			pr-link: Alluxio#18264
			change-id: cid-8124098803c0b05968d67b5ef5c31eed18b69187

### What changes are proposed in this pull request?

A comment in OutStreamTest is useless and need to be removed.

### Why are the changes needed?

Improve code quality.

### Does this PR introduce any user facing changes?

No.

			pr-link: Alluxio#18272
			change-id: cid-602cf0523ed6fbedc30ff448dfb4682006599baa

### What changes are proposed in this pull request?

fix object notserializable bug

### Why are the changes needed?

about issue: Alluxio#18265

### Does this PR introduce any user facing changes?

Please list the user-facing changes introduced by your change, including
  1. change in user-facing APIs
  2. addition or removal of property keys
  3. webui

			pr-link: Alluxio#18266
			change-id: cid-6c74a4a16b470de4a8d7f5a36d2ca87f11d7bb56

Add RESTful API for distributed load.

### Usage:
**SUBMIT:**
description: submit a load job
example:
`http://localhost:28080/v1/load?path=/&opType=submit&partialListing=false&&verify=true&bandwidth=1000&loadMetadataOnly=false&verbose=true&skipIfExists=true`

**STOP:**
description: stop the load job
example:
http://localhost:28080/v1/load?path=/&opType=stop

**PROGRESS:**
description: get the progress of the load job
example:
`http://localhost:28080/v1/load?path=/&opType=progress&progressFormat=text&verbose=true`
			pr-link: Alluxio#18254
			change-id: cid-8a2e4f6747d7ba6cb8c25032cc13ce4ec719da8f