Can't see all files or folders

[dafta]

uMTPrd doesn't detect all files and folders. I'm attaching two files, one is a ls -la output of the home folder, and the other is uMTPrd's debug log.

file-list.txt
umtprd.log

I'm using FunctionFS, and I tried with and without inotify support.

[jfdelnero]

Which version of uMTPrd are you using ?
Not sure which files are not seen. Can you name one for example ?

[dafta]

This has been a long standing issue that I've received multiple complaints about, with multiple versions. Most recently, I tested this with commit 7d7d099 and just now I pulled the latest commits from the repo and tried with the latest version, and the same thing happens.

So in the example above, the .cache folder for instance is shown in the file list, but it doesn't show up over MTP, and the umtprd debug logs don't show .cache as even being detected. Several other folders as well, such as .fonts, bin, and youtube-cast-receiver. Of the files that aren't shown, there's .fonts.conf and .steampath, as far as I can see. It's not limited to just files and folders starting with ., that just happens to be my home folder. Two folders I mentioned don't start with dots, and I've had several users complain about missing files and folders.

Here's a debug log with the newest version of umtprd:

umtprd.log

[jfdelnero]

This might be an access right issue.

Can try to do a "stat .cache" from the same folder ?

Which "user" run the umtp daemon ?

Also please have a look to the configuration file:

# Force the default UID/GID to use for each storage file operations.
# Note : When no default UID/GID are specified the umtprd process UID/GID
#        are used instead.

#default_uid 33

## group: mtp
#default_gid 33

#
# umask value for uploaded folders and files
#
# Uncomment the umask command line if you want to
# override the system default umask value for
# the uploaded folders and files.
#
# umask format description :
#
# umask 022
#       ugo
#
# u : user/owner part of mask
# g : group part of mask
# o : others/not-in-group part of mask
#
# Octal digit definition in umask command :
#
# 0 : Any permission may be set. (Read, Write, eXecute)
# 1 : Setting of execute permission is prohibited. (Read and Write)
# 2 : Setting of write permission is prohibited. (Read and eXecute)
# 3 : Setting of write and execute permissions is prohibited. (Read only)
# 4 : Setting of read permission is prohibited. (Write and eXecute)
# 5 : Setting of read and execute permissions is prohibited. (Write only)
# 6 : Setting of read and write permissions is prohibited. (eXecute only)
# 7 : All permissions are prohibited from being set. (No permission)
#

# Example :
# User   : Read, Write, eXecute.       (0)
# Group  : Read, eXecute but no Write. (2)
# Others : Read, eXecute but no Write, (2)

#umask 022

[jfdelnero]

After a closer look it appears that the issue is definitively generated by this :

According to your file listing this is a link. Is the link valid ?

Anyway the files/folder listing shouldn't stop on such error so i will change the error management at this point.

[jfdelnero]

Can you try to disable the inotify support in the config file ?

# inotify support
# If you want disable the events support (beta), uncomment the following line :

# no_inotify 0x1

( uncomment the "no_inotify" line)

[dafta]

I have already tried it both with inotify on and off, and it is currently off in both of those logs.

As for the symlink, it is valid, but it doesn't matter if the file is a symlink or not. The .cache folder for instance isn't a link, and neither is the .fonts.conf file.

I have already setup the config with the correct UIDs:

# Force the default UID/GID to use for each storage file operations.                                                   
# Note : When no default UID/GID are specified the umtprd process UID/GID                                              
#        are used instead.                                 
                                                           
default_uid 1000                                                                                                       
default_gid 1000

As for stat, there's no difference between folders which can't be seen over MTP such as .cache and youtube-cast-receiver:

$ stat .cache
  File: .cache
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: 259,8   Inode: 22806564    Links: 60
Access: (0755/drwxr-xr-x)  Uid: ( 1000/    deck)   Gid: ( 1000/    deck)
Access: 2025-01-18 00:14:54.906811513 +0100
Modify: 2025-01-18 00:18:46.243715347 +0100
Change: 2025-01-18 00:18:46.243715347 +0100
 Birth: 2022-05-26 13:39:05.313788328 +0200

$ stat youtube-cast-receiver/
  File: youtube-cast-receiver/
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: 259,8   Inode: 23728514    Links: 7
Access: (0755/drwxr-xr-x)  Uid: ( 1000/    deck)   Gid: ( 1000/    deck)
Access: 2025-01-18 00:14:09.619393548 +0100
Modify: 2024-07-15 19:13:05.167457555 +0200
Change: 2024-07-15 19:13:05.167457555 +0200
 Birth: 2024-07-15 19:11:03.694633052 +0200

And folders which can be seen over MTP such as Documents:

stat Documents/
  File: Documents/
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: 259,8   Inode: 22806550    Links: 6
Access: (0755/drwxr-xr-x)  Uid: ( 1000/    deck)   Gid: ( 1000/    deck)
Access: 2025-01-18 00:14:10.616074490 +0100
Modify: 2024-11-24 18:33:14.117548979 +0100
Change: 2024-11-24 18:33:14.117548979 +0100
 Birth: 2022-05-26 13:39:03.463788321 +0200

And there's also no difference between files as well.

.fonts.conf which can't be seen over MTP:

$ stat .fonts.conf 
  File: .fonts.conf
  Size: 111             Blocks: 8          IO Block: 4096   regular file
Device: 259,8   Inode: 22885365    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    deck)   Gid: ( 1000/    deck)
Access: 2025-01-18 00:04:37.354232296 +0100
Modify: 2024-01-17 19:59:18.020327951 +0100
Change: 2024-01-17 19:59:18.020327951 +0100
 Birth: 2024-01-17 19:59:18.020327951 +0100

And .bashrc which can be seen:

$ stat .bashrc 
  File: .bashrc
  Size: 632             Blocks: 8          IO Block: 4096   regular file
Device: 259,8   Inode: 22814058    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    deck)   Gid: ( 1000/    deck)
Access: 2025-01-18 00:04:00.884227996 +0100
Modify: 2024-10-20 17:39:29.511943289 +0200
Change: 2024-10-20 17:39:29.511943289 +0200
 Birth: 2024-10-20 17:39:29.511943289 +0200

[jfdelnero]

The fact that some files are not seen, doesn't mean that they are the problem, this just mean that the folder listing loop was stopped by something else before the end :

The waning message come from fs_entry_stat. And it stop the file listing process with fs_find_first_file/fs_find_next_file functions.

A new check need to be added in the code : if the error come from a link, just skip / ignore it - The target file/folder is probably not present or umtprd doesn't has enough rights to stat/read it.

[dafta]

Hmm yeah I see what you mean. It's not a valid link, you're right, that's my bad. There's two symlinks with a similar name and I checked the other one accidentally earlier.

[jfdelnero]

Thank you for confirming my hypothesis :) . Don't remove the link yet, i will fix the code to manage this kind of error.

[jfdelnero]

@dafta Please get the latest main branch and let me know if the issue is fixed for you.

[dafta]

It seems to be fixed for me. I'll be sure to tell my users to try again and see if it's fixed completely.