Improve packager

[ftsfranklin]

Behavior changes:

• Be more aggressive about ignoring files. It used to ignore files if either the file or its direct dir had a leading dot. Now it checks every ancestor, up to the app_root.
• Copy files to a temporary directory and then builds that.
• Remove the need to halt on carriage return (Carriage return (CR) handling is burdensome #45).
• Fix a regex which (I believe) would have caused user's myfile.cupyd to be deleted during build.
• Add a rudimentary requirements.txt for the packager's libraries.
• Add a rudimentary .gitignore.
• There may be some behavior changes for the prints for "Did not include ...".

Possible future improvements:

• Better way to handle temporary files. Currently, it dumps the files into the build/ folder (the use of which was removed in a previous version), and never cleans up.
• I don't know what to do about copying the file metadata. Will it be okay if the py files get copied into the router with or without global r/w/x?
• The normcase call (to canonicize the path before splitting) MIGHT cause problems with dots (but it shouldn't).
• Clean up shouldinclude2, which was added for how the code used os.walk (instead of glob.iglob('**/*', recursive=True)).
• Allow explicit file list or ignore list to be passed in.
• I wasn't sure whether we could use pathlib.

[amickel]

Hi @ftsfranklin, thanks so much for taking the time to make these improvements! I did notice two issues I would appreciate if you could update the code and re-submit the pull request:

1. Comment out line 199 on make.py.
2. Fix issue with copy_without_cr being passed sub-directories using '/' instead of ''. You can see an example of the issue by building the cli_sample app:
   Packaging cli_sample Traceback (most recent call last): File "tools\bin\package_application.py", line 243, in package_application(sys.argv[1], pkey) File "tools\bin\package_application.py", line 212, in package_application new_paths = copy_app_folder(app_root, temp_root) File "tools\bin\package_application.py", line 79, in copy_app_folder copy_without_cr(fpath, outpath) File "tools\bin\package_application.py", line 88, in copy_without_cr open(dst, 'w', newline='\n') as dstfile: FileNotFoundError: [Errno 2] No such file or directory: 'build\cli_sample\pexpect/ANSI.py' Error packaging cli_sample: Command 'python tools\bin\package_application.py cli_sample' returned non-zero exit status 1.

Thanks!

[ftsfranklin]

Resubmit, instead of adding the changes to the existing PR?

[amickel]

Resubmit, instead of adding the changes to the existing PR?

That works, too!