Add PosixFS (experimental)

modules/ROOT/pages/deployment/storage/general-considerations.adoc

@@ -9,11 +9,11 @@
 
 Note that depending on the storage used, special settings can be defined to optimize resources for memory and performance.
 
-For a quick and rough overview, the following storages are mapped against an expected sizing. Note the way that Infinite Scale is setup like xref:depl-examples/bare-metal.adoc[binary] or xref:deployment/container/orchestration/orchestration.adoc[orchestration] with its variants usually comes along with the expected sizing:
+For a quick and rough overview, the following storages are mapped against an expected sizing. Note the way that Infinite Scale is setup like as xref:depl-examples/bare-metal.adoc[binary] or via an xref:deployment/container/orchestration/orchestration.adoc[orchestration] with its variants usually comes along with the expected sizing or usecase:
 
 {empty} +
 
-[role=center,width=100%,cols="25%,45%,45%",options="header"]
+[role=center,width=100%,cols="20%,45%,45%",options="header"]
 |===
 | Size
 | Case
@@ -31,6 +31,11 @@ For a quick and rough overview, the following storages are mapped against an exp
 | Large enterprises and hosters
 | S3 for blobs +
 NFS for metadata and other data
+
+| --
+| Special usecase +
+Shared access to the underlying filesystem by Infinite Scale and ordinary OS users or services
+| PosixFS (experimental)
 |===
 
 [NOTE]
@@ -39,9 +44,11 @@ NFS for metadata and other data
 * The levels of space requirement described below give an indication of what to expect, but never a number.
 ====
 
+IMPORTANT: See the note for distinguishing posix vs PosixFS in the xref:deployment/storage/posixfs.adoc[PosixFS] documentation.
+
 == Write Consistency
 
-Independent of the storage used and how many instances of the xref:{s-path}/storage-users.adoc[storage-users] service are running, Infinite Scale has mechanisms implemented to guarantee consistent writing to the backend for distributed setups. In any case, a file ID based on UUIDv4 is created to guarantee the uniqueness of file names. Proper file locking is implemented for writing metadata.
+Independent of the storage used and how many instances of the xref:{s-path}/storage-users.adoc[storage-users] service are running, Infinite Scale has mechanisms implemented to guarantee consistent writing to the backend for distributed setups. For all storage types except PosixFS, a file ID based on UUIDv4 is created to guarantee the uniqueness of file names. Proper file locking is implemented for writing metadata.
 
 == Important Data Distribution Notes
 
@@ -167,11 +174,18 @@ Alternatively NFS_4
 | High
 |===
 
+== PosixFS
+
+PosixFS is currently experimental only and should not be used in production environments.
+
+For details on PosixFS including setup and configuration notes see the xref:deployment/storage/posixfs.adoc[PosixFS] documentation.
+
 == Server Resource Optimisation
 
 Depending on the storage connected and the servers capabilities, Infinite Scale can be optimized using the servers resources. The relevant environment variable to configure this is:
 
-`STORAGE_USERS_OCIS_MAX_CONCURRENCY`
+`STORAGE_USERS_OCIS_MAX_CONCURRENCY` +
+(or `OCIS_MAX_CONCURRENCY` when defined on a global basis)
 
 The value to consider and only as a rule of thumb is based on how much CPU's and memory the server has the instance of the xref:{s-path}/storage-users.adoc[storage-users] service is running on, which kind of storage, POSIX or S3 is used for blobs and what you want to achieve.
 

modules/ROOT/pages/deployment/storage/posixfs.adoc

@@ -0,0 +1,102 @@
+= PosixFS
+:toc: right
+:toclevels: 2
+:description: Infinite Scale can store metadata and files on posix based filesystems with shared access. This document gives some general information and considerations for the use of PosixFS.
+
+== Introduction
+
+{description} See the xref:deployment/storage/general-considerations.adoc#posixfs[General Storage Considerations] in the PosixFS section for more details.
+
+IMPORTANT: The PosixFS storage driver integration is an experimental feature and should not used in production.
+
+[IMPORTANT]
+====
+POSIX is not equal to PosixFS
+
+* *POSIX* defines standards for operating systems in general and filesystems in particular. A native posix compliant filesystem is for example ext3/4, zfs or NFS, while S3 or CIFS are not and need additional abstraction layers to be posix compliant with some exceptions.
+
+* *PosixFS* is an Infinite Scale storage driver implementation that can use fully compliant posix filesystems with shared access. It requires some mandatory posix filesystem features to be available like handling extended attributes and notification processes. Note that when mounting an external filesystem like SMB into a posix compliant filesystem, PosixFS may not work due to incompatibilities.
+====
+
+== PosixFS Overview
+
+When PosixFS is configured, a user may not only access data via Infinite Scale as usual, but also via the filesystem directly. This concept allows, for example, services like printers, scanners to write data to a location Infinite Scale has access to and treats these files like they were managed through Infinite Scale directly. On the other hand, users or services can access data for further processing as they would normally do via the filesystem without going through Infinite Scale.
+
+Comparison of private vs shared storage in Infinite Scale::
+* When *private storage* is configured: +
+Only Infinite Scale can manage data. It is in full control over the data and structure, which is not easily human readable.
+* When *shared storage* is configured: +
+Infinite Scale is not in full control over the underlying data and structure. It needs to be notified of changes and has to add required metatdata via extended attributes to make the process flow work like in a private storage.
+
+== Limitations
+
+* A filesystem must be posix compliant and must fully support extended attributes.
+* Only `inotify` and `GPFS file system change notification` notification methods are currently supported.
+* Versioning is not supported yet.
+* The spaces are folders in the filesystem and named after the UUID, not the real space name
+* No support for CephFS or other mounted filesystems like NFSv3 or SMB yet. +
+NFSv4.2 is working. This filesystem provides sufficient extended attribute capabilities. Check the documentation of your storage provider for details.
+* Postprocessing like antivirus scanning does not get triggered for file actions that are not initiated by Infinite Scale.
+* Though there are more options available, we recommend using a group based access for the shared storage. This means, that both Infinite Scale and the accessing users/services must use a common group. The default umask in the directory used has to allow group writing all over the place.
+
+== Setup
+
+Note that it is possible to use different storage drivers in the same Infinite Scale installation. For example, one is set up for one space running on PosixFS Storage Driver while all others run Decomposed FS with a private storage. This is an enhanced configuration and requires multiple instances of the xref:{s-path}/storage-users.adoc[Storage-Users] service running, individually configured. The details are not covered here.
+
+=== Prerequisites
+
+To use the PosixFS Storage Driver, the following prerequisites have to be fulfilled:
+
+* There must be storage available to store meta data (extended attributes) and files, available under a root path.
+* When using the `inotify` mechanism, `inotifywait` is required and has to be installed if not present.
+* The storage root path must be writeable and executable by the same user or group Infinite Scale is running under.
+* Infinite Scale must be at version 7 or higher.
+* You must use `nats-js-kv` as cache service, see xref:deployment/services/caching.adoc[Caching and Persistence] for more details.
+
+=== Configuration
+
+The following is an example configuration you can derive from with environment variables that configures Infinite Scale to use the PosixFS Storage Driver for all spaces. It works with Personal and Project Spaces. Use these settings in your deployment and replace `<local-path>` with your root entry point where Infinite Scale will access data.
+
+[source,yaml]
+----
+STORAGE_USERS_DRIVER="posix"
+STORAGE_USERS_POSIX_ROOT=<local-path>
+STORAGE_USERS_POSIX_WATCH_TYPE="inotifywait"
+STORAGE_USERS_ID_CACHE_STORE="nats-js-kv"
+STORAGE_USERS_ID_CACHE_STORE_NODES="localhost:9233"
+STORAGE_USERS_POSIX_USE_SPACE_GROUPS="true"          
+----
+
+=== GPFS Specifics
+
+When using GPFS as the underlying filesystem, the machine running the xref:{s-path}/storage-users.adoc[Storage-Users] service needs to have the GPFS filesystem mounted locally. The mount path for Infinite Scale is defined via the `STORAGE_USERS_POSIX_ROOT` environment variable and can be any directory inside the mount as entry point.
+
+Other than that there a few other points to consider:
+
+==== Extended Attributes
+
+As described above, metadata is stored as extended attributes of the entities and thus is subject to their limitations. In GPFS, extended attributes are first stored in the inode itself but can then also use an overflow block which is at least 64KB and up to the metadata block size. Inode and metadata block size should be chosen accordingly.
+
+==== FS Watcher
+
+The PosixFS Storage Driver supports two different watchers for detecting changes to the filesystem. The watchfolder watcher is better tested and supported at that point.
+
+===== GPFS File Audit Logging
+
+The `gpfsfileauditlogging` watcher tails a GPFS file audit log and parses the JSON events to detect relevant changes.
+
+[source,yaml]
+----
+STORAGE_USERS_POSIX_WATCH_TYPE="gpfsfileauditlogging"
+STORAGE_USERS_POSIX_WATCH_PATH="/path/to/current/audit/log"
+----
+
+===== GPFS Watchfolder
+
+The `gpfswatchfolder` watcher connects to a kafka cluster which is being filled with filesystem events by the GPFS watchfolder service.
+
+[source,yaml]
+----
+STORAGE_USERS_POSIX_WATCH_TYPE="gpfswatchfolder"
+STORAGE_USERS_POSIX_WATCH_PATH="fs1_audit"         # the kafka topic to watch
+----

modules/ROOT/pages/maintenance/b-r/backup.adoc

@@ -40,9 +40,9 @@ Keep in mind that you always have to back up all data sets consistently:
 
 {empty}
 
-(2) ... For POSIX-only environments, the location for blobs and metadata is the same. When using S3, the location differs. The backup process must include the metadata, for details see xref:prerequisites/prerequisites.adoc#filesystems-and-shared-storage[Filesystems and Shared Storage].
+(2) ... For POSIX-only environments, the location for blobs and metadata is the same. When using S3, the location differs. The backup process must include the metadata to be complete, for details see xref:prerequisites/prerequisites.adoc#filesystems-and-shared-storage[Filesystems and Shared Storage].
 
-As a rule of thumb, consider also to back up the Infinite Scale binary or the container used. This avoids difficulties when restoring the instance and possibly using an updated Infinite Scale version compared to the version used when backing up. Restoring and updating should always be different tasks.
+As a rule of thumb, consider also to back up the Infinite Scale binary or the container used including the configuration. This avoids difficulties when restoring the instance and possibly using an updated Infinite Scale version compared to the version used when backing up. Restoring and updating should always be different tasks.
 
 == Pure POSIX Setup
 

modules/ROOT/pages/maintenance/b-r/restore.adoc

@@ -28,7 +28,7 @@ When it comes to restoring data, keep in mind that you always have to restore al
 
 {empty}
 
-(2) ... For POSIX-only environments, the location for blobs and metadata is the same. When using S3, the location differs. The restore process must include the metadata, for details see xref:prerequisites/prerequisites.adoc#filesystems-and-shared-storage[Filesystems and Shared Storage].
+(2) ... For POSIX-only environments, the location for blobs and metadata is the same. When using S3, the location differs. The restore process must include the metadata to be complete, for details see xref:prerequisites/prerequisites.adoc#filesystems-and-shared-storage[Filesystems and Shared Storage].
 
 Valid for all restore procedures:
 

modules/ROOT/pages/prerequisites/prerequisites.adoc

@@ -280,12 +280,17 @@ Metadata is stored as {messagepack}[messagepack] files. Messagepack files have a
 
 === Available Filesystem Drivers
 
+See the xref:deployment/storage/general-considerations.adoc[General Storage Considerations] documentation for more details on individual filesystem drivers.
+
 The ocis filesystem driver::
 When the `ocis` driver is used, blobs and metadata must be on a POSIX-compliant filesystem. This driver decomposes the metadata and persists it in a POSIX filesystem. Blobs are stored on the filesystem as well. This layout makes extensive use of symlinks. A filesystem like xfs or zfs without practical inode size limitations is recommended. A further integration with file systems like CephFS or GPFS is under investigation.
 
 The s3ng filesystem driver::
 When the `s3ng` driver is used, blobs reside on a S3 bucket and the metadata will be stored on a POSIX-compliant filesystem which needs to be provisioned. As this POSIX-compliant filesystem usually needs to be mounted on several Infinite Scale instances like when deploying with a xref:deployment/container/orchestration/orchestration.adoc[container orchestration], consider using NFS for storing this metadata. This splitting is necessary for performance reasons.
 
+The PosixFS filesystem driver (experimental)::
+When the `posixfs` driver is used, you can connect to fully compliant posix filesystems with shared access. It requires some mandatory posix filesystem features to be available like handling extended attributes and notification processes. This also differentiates it from other drivers that use messagepack for metadata. Note that extended attributes need to be included in the backup strategy.
+
 Other drivers can be used too like for the Ceph or EOS filesystem, but no support can be given because they are not developed or maintained by ownCloud. 
 
 === Supported POSIX-compliant Filesystems
@@ -348,15 +353,15 @@ The storage system must not impose a cap on the input/output operations per seco
 
 == Compatible Apps and Clients
 
-At the moment, spaces are supported with the new ownCloud Web interface and the Desktop app version 3.0.
+Spaces are supported with the ownCloud Web interface and since the Desktop app version 3.0.
 
 * Web (embedded in Infinite Scale)
-* Desktop app 3.0
+* Desktop app > 3.0
 
-Spaces for the mobile apps are available in versions:
+Spaces for the mobile apps are available since versions:
 
-* iOS app 12.0
-* Android app 4.0
+* iOS app > 12.0
+* Android app > 4.0
 
 == Supported Languages
 

modules/ROOT/partials/nav.adoc

@@ -10,6 +10,7 @@
 *** xref:deployment/storage/general-considerations.adoc[General Storage Considerations]
 *** xref:deployment/storage/nfs.adoc[Network File System]
 *** xref:deployment/storage/s3.adoc[S3]
+*** xref:deployment/storage/posixfs.adoc[PosixFS]
 ** xref:deployment/general/general-info.adoc[General Information]
 ** xref:deployment/general/ocis-init.adoc[The ocis init Command]
 ** xref:deployment/container/orchestration/orchestration.adoc[Container Orchestration]