Added readme.

README.md

@@ -28,7 +28,8 @@ There are data structures for almost all PCBoard data structures so making a BBS
   This is intended - all modern BBSes look the same. You can choose from a wide range of plugins and configuration options.
   Use it!
 
-[GettingStarted](docs/gettingstarted.md)
-[Differences](docs/differences.md)
-[PPL](docs/ppl.md)
+# Topics
+* [GettingStarted](docs/gettingstarted.md)
+* [Differences](docs/differences.md)
+* [PPL](docs/ppl.md)
 

docs/differences.md

@@ -0,0 +1,62 @@
+# Differences to PCBoard
+
+The goal is to be as compatible as possible but enhancing PCBoard for the moderen age implies changes that may break compatibiltiy.
+
+* All config files are in .toml format and can be changed with any text editor
+  * Old config files are created automatically for old .PPE compatibility files but some PPEs might break. (Report please)
+  * TOML formats include: Bullettins, Script Questionnaires and Menus
+  * Exceptions so far: Trashcan file 
+* Long file name support! yeah! (However old config files have limits)
+  * Applies to old PPEs as well they got their limits extended, unless they check for file lengths themself.
+* Message base format changed to JAM
+  * Planned is to support multiple message base formats
+  * Means all PPEs/tools break that may read the old format
+* DIR files are no longer used. Just throw the files in the directory, icy board takes care of everything.
+
+## CP437 is dead
+
+All files should be UTF-8. The importer automatically converts files to UTF-8.
+
+However it's possible to use CP437 files with IcyBoard. Icy Board differenciates UTF-8 from CP437 with the UTF8 BOM.
+All files without  FF EF FE at the start are treated as UTF8. So it's no hard requirement. Copy old DOS files and they will show up fine regardless of CP437 or UTF-8 used.
+
+So it's possible to use a modern (non CP437) text editor to alter the BBS files.
+
+* Config files, Menus, (in general: non display text files) are ALWAYS UTF-8.
+* UTF8->CodePage 437 chars will be translated by the table found here: <https://en.wikipedia.org/wiki/Code_page_437>
+
+## Command changes
+
+* No more hard coded commands. They're now all configurable. (however all pcb commands are built-in but can now be configured)
+
+## PPE/Menus
+
+* `.MNU` files are converted to a new .toml format (in the assumption that no `.PPE` will handle `.MNU` files)
+* PCBoard is now case sensitive on unix but that does NOT apply to `.PPE` files. Note: May change for new `.PPE` files but will always remain for old ones for compatiblity reasons.
+
+## Trashcan
+
+A simple text file read line by line. If a line starts with '#' it's treated as a comment.
+Leading & traling whitespaces are ignored
+
+
+## Enhancements
+
+* Access System is more complex. The old one had user levels. An access now consists of a combination of:
+  * security level
+  * group (like unix groups)
+  * age 
+* Added more trashcans: email/passwords
+* New 'vip' users vip_users.txt (same as trashcan). But a list of users which the sysop gets a notification for logon (from RemoteAccess)
+* Surveys (questionnaires) header can now have a different header length than 5 lines
+Header is separated from questions with a line starting by "*****". Note that the pcb importer inserts a "*****" line after the 5th during import.
+
+## Planned Enhancements/Discussion 
+
+* Access system time limit - so it's only open at certain days & times - however only command where that makes sense to me is the sysop page.
+But I like the RA "DayTimes" system.
+
+## New @ Sequences
+
+@GFXMODE@ = Off, Ansi, Avatar or Rip (for now) depending on the current graphics mode
+@NUMAREA@ = Number of message areas in current conference

docs/gettingstarted.md

@@ -0,0 +1,77 @@
+# Getting Started
+
+First grab a release for your operating system:
+https://github.com/mkrueger/icy_board/releases/latest
+
+Or build from source. Building from source is easy as well.
+It's installing a [rust development](https://www.rust-lang.org/tools/install) environment and just run
+
+`cargo build --release`
+
+If something is missing on your system cargo build will tell you. If you know hat a development environment is it should be straightforward.
+
+I develop this software on linux - next time I set up I'll add a more detailed description.
+
+# Startup Icy Board
+
+I recommend putting the bin/ directory in the path but you can just `cd bin` for now.
+
+First create a new BBS: `./icbsetup create FOO`
+Then start it: `./icyboard FOO`
+
+This will fire up a new call waiting screen where you can log in as sysop. 
+
+# Tools
+
+* Most important is ICBSetup - that contains all options for IcyBoard. It's a mess!
+* ICBText - there you can edit all text messages. This is the main way of extending IcyBoard through PPEs
+* ICBSysMgr - that let's you edit the users.
+
+# Directory Layout
+
+I tried to simplify the PCBoard system a bit but it has limits.
+
+I designed IcyBoard for using relative paths. However absolute ones can be used. Relative path root is always where the main icboard.toml is. Regardless of file position.
+This makes it easier to move files around - if needed and cut & paste etc.
+
+Basically the file Layout is:
+| File/Dir | Description|
+| --- | --- |
+|icyboard.toml | Main Config File |
+|icyboard.log | Log File |
+|art/| All ANSIS go in there | 
+|art/help/| Help Files | 
+|config/| All config files are here | 
+|conferences/| Conference data (files/messages) |
+|tmp/| Generated Files for backwards compatiblity |
+
+The log file is very important. If something goes wrong it's likely that the log file tells you why.
+
+## Config files
+
+| File | Description|
+| --- | --- |
+|commands.toml | All Commands |
+|conferences.toml | Conference data |
+|groups| Unix Like /etc/gorups file | 
+|icbtext.toml| Contains all Icy Board System Messages | 
+|languages.toml| Language descriptions (Date Formats, yes/no characters & localized icbtext.toml locations) | 
+|protocols.toml| List & Description of available transfer protocols |
+|security_levels.toml| Security Levels & Limits |
+|users.toml| Contains registered all User Records |
+|tcan_user.txt| Forbidden user names |
+|tcan_passwords.txt| Forbidden user passwords |
+|tcan_email.txt| Forbidden emails |
+|tcan_uploads.txt| Forbidden upload file names |
+|vip_user.txt| Users where the sysop is informed about a login |
+
+*NOTE: The location & name of all files can be changed in the main icboard.toml.*
+
+# ART files
+
+It's recommended to use .pcb, .ans, .rip, .asc extensions instead of the old *G, *R sheme. 
+This makes it easier to draw files with an ansi drawing tool as well. And file name lengths ar no longer
+an issue.
+Files can either be CP437 or UTF-8 - IcyBoard will do all conversions automatically. Note that UTF-8 requires the UTF-8 BOM. This is by design it's the only way to make a fast and correct decision about the file encoding.
+
+Note: UTF-8 is recommended for everything.

docs/ppl.md

@@ -0,0 +1,204 @@
+
+# PPL
+
+IcyBoard features a rewriten engine for ppl execution, a compiler and a decompiler.
+
+Features:
+
+* A compiler (pplc) that compiles UTF-8/CP437 files to output CP437 PPEs
+* A decompiler (ppld) that decompiles all old .PPE files (up to PCBoard 15.4 PPEs) 
+  - Reconstructs PPL all old statements 
+* A language server that provides developer functionality in any editor that supports lsp
+  - Included VS Code Extension (.vsix) for easy installation - just dnd it into the vs code extensions panel.
+
+## What works
+
+* Both compiler & decompiler is DONE. I would say it's better than everything we had back in the 90'.
+Everything that doesn't work is a bug - please report issues.
+* The decompiler should be able to decompile any existing PPE file and trick out the anti decompiliation tricks that were common these days :). Note that it's not designed to decompile the new 400 PPEs.
+* Compiler should be able to parse a PPS and generate running PPE files
+  - There are slight differences to PPLC - the new one is more strict. Issues should be easy fixable
+  - Be prepared for tons of warnings of non trivial .PPS files. The old PPLC hasn't had much error checks. In doubt I added a warning instead.
+* IcyBoard should be able to run most PPE files
+  - PPE data files can be converted to UTF8 with (icbsetup ppe-convert <PATH>) but backup all files first
+  - ppe-convert can take a <FILENAME> to convert the single file to UTF8
+  - WARNING: Handle ppe-convert with care - can potentially destroy things. Convert one PPE after another.
+  - No need to convert PPE - CP437 works, just consider that - I do it because no modern editor supports CP437 anymore.
+* LSP should provide highlighting, help, find all refs/goto definition and a basic code completion 
+
+### Decompiler
+
+First Decompiler was based upon ppld. Find the original code here:
+https://github.com/astuder/ppld
+
+Much effort was done for implementing the decompiler. Existing PPEs may need to be altered for IcyBoard or at least analyzed so being able to decompile
+the old PPEs is important for the project.
+
+The current Decompiler is completely rewritten and uses a ppl machine language - which it can disassemble - to reconstruct a PPL AST.
+
+* PPE 3.40 Support
+* Full reconstruction of IF/THEN, SWITH, WHEN etc.
+* It tries to do some name guessing based on variable usage.
+
+```text
+Usage: ppld <file> [-r] [-d] [-o] [--style <style>]
+
+PCBoard Programming Language Decompiler
+
+Positional Arguments:
+  file              file[.ppe] to decompile
+
+Options:
+  -r, --raw         raw ppe without reconstruction control structures
+  -d, --disassemble output the disassembly instead of ppl
+  -o, --output      output to console instead of writing to file
+  --style           keyword casing style, valid values are u=upper (default),
+                    l=lower, c=camel
+  --help, help      display usage information
+```
+
+The dissamble output can be used to see what the compilers are generating and for debugging purposes.
+
+### Compiler
+
+Supports up to 15.4 PPL (1.0 -> 3.40 PPE format)
+
+Should be compatible to the old PCB compiler with some slight differences (see PPL differences)
+
+The compiler decides itself if uservars are generated or not (so --novars is no longer needed)
+
+pplc has following options:
+
+```text
+Usage: pplc <file> [-d] [--nowarnings] [--version <version>] [--lang-version <lang-version>] [--cp437 <cp437>]
+
+PCBoard Programming Language Compiler
+
+Positional Arguments:
+  file              file[.pps] to compile (extension defaults to .pps if not
+                    specified)
+
+Options:
+  -d, --disassemble output the disassembly instead of compiling
+  --nowarnings      don't report any warnings
+  --version         version number for the compiler, valid: 100, 200, 300, 310,
+                    330, 340, 400 (Default)
+  --lang-version    version number for the language (defaults to `version`)
+  --cp437           specify the encoding of the file, defaults to autodetection
+  --help, help      display usage information
+
+As default the compiler takes UTF8 input - DOS special chars are translated to CP437 in the output.
+```
+
+Note:  All old DOS files are usually CP437 - so it's recommended to use --cp437 for compiling these.
+
+#### PPL differences
+
+The aim is to be as compatible as possible.
+
+* Added keywords that are invalid as identifiers (but are ok for labels):
+  ```LET```, ```IF```, ```ELSE```, ```ELSEIF```, ```ENDIF```, ```WHILE```, ```ENDWHILE```, ```FOR```, ```NEXT```, ```BREAK```, ```CONTINUE```, ```RETURN```, ```GOSUB```, ```GOTO```, ```SELECT```, ```CASE```, ```DEFAULT```, ```ENDSELECT```
+
+I think it improves the language and it's open for discussion. Note that some aliases like "quit" for the break keyword is not a keyword but is recognized as 'break' statement. I can change the status of a keyword so it's not a hard limit - as said "open for discussion".
+
+* Added ```€``` as valid identifier character. (for UTF8 files)
+* Return type differences in function declaration/implementation is an error, original compiler didn't care.
+
+
+#### PPL 4.0
+
+New Constructs (Language Version 350):
+
+New loops
+``` REPEAT ... UNTIL [CONDITION] ``` Statement
+``` LOOP ... ENDLOOP``` Statement
+
+Variable initializers:
+
+``` TYPE VAR=[INITIALIZER]``` Statement
+
+It's possible to initialize dim expressions as well:
+
+``` TYPE VAR={ expr1, expr2, ..., exprn }``` means:
+
+```PPL
+TYPE VAR(n)
+VAR(0) = expr1
+...
+VAR(n - 1) = exprn
+```
+
+Operator Assignment for binary (non condition operators):
+
+Example:
+``` A += 1``` Statement
+
+Works for ```+-*/%``` and ```&|```
+
+Return can now return values inside functions:
+
+``` RETURN expr``` Statement
+
+1:1 same semantic as:
+
+```PPL
+FUNC_NAME = expr
+RETURN 
+```
+
+No more brackets needed for if or while statements!
+
+Example:
+```PPL
+IF A <> B THEN
+...
+ENDIF
+
+WHILE IsValid() PRINTLN "Success."
+```
+
+Note: With "lang" version >=350 'Quit' and 'Loop' are no longer synonyms for 'break' and 'continue'. Existing sources should be easily adapted.
+But never saw them in the wild.
+
+##### Language Version 400
+
+Language Version breaks compatibility with older PCBoards.
+
+WARNING: 400 is not yet finished so expect to have to recompile in the final version.
+
+Changes:
+
+* (), {} and [] is different. [] is for indexer expressions it's encouraged to use that for arrays. And '{', '}' is exclusive for array initializers.
+
+Member references. 400 introduces new BBS types:
+* Conference, MessageArea, FileArea
+
+For example:
+```PPL
+CONFERENCE CUR = CONFINFO(i) 
+
+IF CUR.HasAccess() 
+  PRINTLN CUR.Name
+```
+
+It's basically pre defined objects. This change requires a slight change in PPE.
+
+* Pre defined functions can now have overloads. See "CONFINFO" from above. CONFINFO has two versions: One old one with 2 parameters and the new one with 1.
+The new version returns a CONFERENCE object where the old one basically an "Object" of varying types, depending on the requested conference field.
+
+API still TBD. CONFINFO(i) which returns the CONFERENCE is the only new function so far. However it contains some member functions for message & file areas. All BBS "objects" should be accessible through PPL. It should no longer be needed to gather information through manual reading of config files anymore.
+
+## Building & Running
+
+* Get rust on your system <https://www.rust-lang.org/tools/install>
+
+```bash
+cd PPLEngine
+cargo build -r
+```
+
+```bash
+cd target/release
+./ppld [PPEFILE]
+./pplc [PPLFILE]
+```