Merge pull request #4 from Yattabyte/beta

Beta

.gitignore

@@ -36,9 +36,11 @@ x64/
 x32/
 Documentation/
 .vs/
-src/nStaller/nStaller.dir/
-src/nUpdater/nUpdater.dir/
+src/Installer/Installer.dir/
 src/nSuite/nSuite.dir/
+src/Uninstaller/Uninstaller.dir/
+src/Unpacker/Unpacker.dir/
+src/Updater/Updater.dir/
 app/
 
 # Other

CMakeLists.txt

@@ -1,3 +1,6 @@
+####################
+### nSuite Tools ###
+####################
 cmake_minimum_required(VERSION 3.0)
 project(nStallerTools)
 
@@ -8,19 +11,29 @@ set (PROJECT_BIN ${CMAKE_SOURCE_DIR})
 set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /MP")
 
 # Include directories of dependencies for entire project
-include_directories ( 	${LZ4_DIR}/lib/
-						${CORE_DIR}		)
+include_directories( 
+	${LZ4_DIR}/lib/
+	${CORE_DIR}		
+)
+
+# Add libraries common throughout entire project
+link_libraries(	
+	debug		${LZ4_DIR}/x64_Debug/liblz4_static.lib
+	optimized	${LZ4_DIR}/x64_Release/liblz4_static.lib
+	Gdiplus.lib
+)
 
-link_libraries 	(	debug		${LZ4_DIR}/x64_Debug/liblz4_static.lib
-					optimized	${LZ4_DIR}/x64_Release/liblz4_static.lib	)
-
 # add all sub-projects and plugins here
-add_subdirectory( "src/nStaller" )
-add_subdirectory( "src/nUpdater" )
+add_subdirectory( "src/Installer" )
+add_subdirectory( "src/Uninstaller" )
+add_subdirectory( "src/Unpacker" )
+add_subdirectory( "src/Updater" )
 add_subdirectory( "src/nSuite" )
+
+# Visual studio specific setting: make nSuite the startup project
 set_property(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} PROPERTY VS_STARTUP_PROJECT nSuite)
 
 # Enable folder structure
-set_property (GLOBAL PROPERTY USE_FOLDERS ON)
+set_property(GLOBAL PROPERTY USE_FOLDERS ON)
 # ALL_BUILD, ZERO_CHECK, and other build functions' folder
-set_property (GLOBAL PROPERTY PREDEFINED_TARGETS_FOLDER Build_Functions)
+set_property(GLOBAL PROPERTY PREDEFINED_TARGETS_FOLDER Build_Functions)

README.md

@@ -1,38 +1,36 @@
-# nStaller Tools
+# nSuite Directory Tools
 
-This project is both a library and a toolset that allows developers to generate and distribute portable installers for a given input directory, as well as diff the contents of 2 input directories into a single patch file.
-The library includes functions to compress/decompress and diff/patch both files and directoires.
-The toolset wraps this functionality into a few example programs.
+This toolset provides users with the ability to package and diff directories and files.
 
-## nSuite.exe
-The nSuite tool is intended to be used by developers or those who wish to package/diff/distribute one or many files. It is run by command-line, and requires one of the following sets of arguments to be fulfilled:
-- #### `-installer -src=<path> -dst=<path>`
-  - Packages + compress an entire **source** directory into a **destionation** installer *.exe file* (filename optional)  
-
-- #### `-pack -src=<path> -dst=<path>`
-  - Same as the installer, though doesn't embed into an installer, just a *.npack* file (filename optional)
-  - Package files can be read-through by the diffing command, so many versions of a directory can be easily stored on disk in single snapshots.
+
+## Packaging
+nSuite can package directories in 3 ways:
+- A fully fledged installer with a GUI (Windows)
+  - Customizable by writing attributes into a manifest file
+  - Generates an uninstaller (adds it to the registry)
+  - .npack file embedded within
 
-- #### `-unpack -src=<path> -dst=<path>`
-  - Decompresses the files held in the **source** *.npack* file, dumping into the **destination** directory.
+- A lightweight portable package/installer
+  - Extracts to a folder in the directory it runs from
+  - Runs in a terminal, no user input
+  - Doesn't modify registry - no uninstaller
+  - .npack file embedded within
 
-- #### `-diff -old=<path> -new=<path> -dst=<path>`
-  - Finds all the common, new, and old files between the **old** and **new** directories. All common files are analyzed bytewise for their differences, and patch instructions are generated. All instructions are compressed and stored in a **destination** *.ndiff* file. ***All*** files are hashed, to ensure the right versions of files are consumed at patch-time.
-  - Can use *.npack* files as the **old** ***or*** **new** directories (serving as snapshots). Diffing packages means storing less files and folders across multiple versions on disk -> just 1 snapshot per version.
+- A .npack file
+  - Can be unpacked using nSuite
+
 
- - #### `-patch -src=<path> -dst=<path>`
-   - Uses a **source** *.ndiff* file and executes all the instructions contained within, patching the **destination** directory. All files within the directory are hashed, and must match the hashes found in the patch. Additionally, the post-patch results must match what's expected in the hash. If any of the strict conditions aren't met, it will halt prior to any files being modified (preventing against file corruption).
-
-
-## Installer
-The installer tool is a portable version of the unpack command, and is generated by nSuite. Each one will have a custom *.npack* file embedded within. The installer is very basic and installs to the directory it runs from.
+## Diffing
+nSuite can also be used to generate patch files. These can be applied to a directory using nSuite, or by using our stand-alone updater tool (also provided).
+
+The updater tool automatically applies all .ndiff files it can find next to it, and if successfull, deletes them afterwards. This tool is a naiive implementation of an updater, and would ideally be expanded on by other developers for real-world use. 
 
-## Updater
-The updater tool is a portable version of the patch command. It automatically applies all *.ndiff* files it can find, and if successfull, deletes them after. This tool is a naiive implementation of an updater, and would ideally be expanded on by other developers. For instance, if patches were found that would modify an app from v.1 -> v.2 -> v.3 -> v.1, the updater won't try to stop at v.3 (as there are no version headers applied to patches). Further, this updater cannot connect to any servers to fetch patch data, but that would be the next logical step after implementing versioning. 
+The tool and diff files should be kept at the root of an affected directory. It will attempt to apply all patches it can find, even if the patched version is technically 'older'.
 
 # Dependencies/Requirements
- - 64-bit only
- - Might only work in Windows
+ - C++ 17
+ - 64-bit
+ - Windows 7/8/10
  - Uses [CMake](https://cmake.org/)
  - Requires the [LZ4 - Compression Library](https://github.com/lz4/lz4) to build, but **does not** come bundled with it
- - Using BSD-3-Clause license
+ - Using BSD-3-Clause license

src/BufferTools.cpp

@@ -1,4 +1,5 @@
 #include "BufferTools.h"
+#include "Common.h"
 #include "Instructions.h"
 #include "Threader.h"
 #include "lz4.h"
@@ -7,32 +8,40 @@
 
 bool BFT::CompressBuffer(char * sourceBuffer, const size_t & sourceSize, char ** destinationBuffer, size_t & destinationSize)
 {
-	// Allocate enough room for the compressed buffer (2 size_t bigger than source buffer)
-	destinationSize = (sourceSize * 2ull) + size_t(sizeof(size_t));
-	*destinationBuffer = new char[destinationSize];
-
-	// First chunk of data = the total uncompressed size
-	*reinterpret_cast<size_t*>(*destinationBuffer) = sourceSize;
+	// Pre-allocate a huge buffer to allow for compression OPS
+	char * compressedBuffer = new char[sourceSize * 2ull];
 
-	// Increment pointer so that the compression works on the remaining part of the buffer
-	*destinationBuffer = reinterpret_cast<char*>(*destinationBuffer) + size_t(sizeof(size_t));
-
-	// Compress the buffer
+	// Try to compress the source buffer
 	auto result = LZ4_compress_default(
 		sourceBuffer,
-		*destinationBuffer,
+		compressedBuffer,
 		int(sourceSize),
-		int(destinationSize - size_t(sizeof(size_t)))
+		int(sourceSize * 2ull)
 	);
 
-	// Decrement pointer
-	*destinationBuffer = reinterpret_cast<char*>(*destinationBuffer) - size_t(sizeof(size_t));
-	destinationSize = size_t(result) + sizeof(size_t);
+	// Create the final buffer (done separate b/c we now know the final reduced buffer size)
+	constexpr size_t HEADER_SIZE = size_t(sizeof(size_t));
+	destinationSize = HEADER_SIZE + size_t(result);
+	*destinationBuffer = new char[destinationSize];
+	char * HEADER_ADDRESS = *destinationBuffer;
+	char * DATA_ADDRESS = HEADER_ADDRESS + HEADER_SIZE;
+
+	// Header = the total uncompressed size
+	*reinterpret_cast<size_t*>(HEADER_ADDRESS) = sourceSize;
+
+	// Data = compressed source buffer
+	std::memcpy(DATA_ADDRESS, compressedBuffer, size_t(result));	
+	delete[] compressedBuffer;
+
 	return (result > 0);
 }
 
 bool BFT::DecompressBuffer(char * sourceBuffer, const size_t & sourceSize, char ** destinationBuffer, size_t & destinationSize)
 {
+	// Ensure buffer at least *exists*
+	if (sourceSize <= size_t(sizeof(size_t)) || sourceBuffer == nullptr)
+		return false;
+
 	destinationSize = *reinterpret_cast<size_t*>(sourceBuffer);
 	*destinationBuffer = new char[destinationSize];
 	auto result = LZ4_decompress_safe(
@@ -48,7 +57,7 @@ bool BFT::DecompressBuffer(char * sourceBuffer, const size_t & sourceSize, char
 bool BFT::DiffBuffers(char * buffer_old, const size_t & size_old, char * buffer_new, const size_t & size_new, char ** buffer_diff, size_t & size_diff, size_t * instructionCount)
 {
 	std::vector<InstructionTypes> instructions;
-	instructions.reserve(std::max(size_old, size_new) / 8ull);
+	instructions.reserve(std::max<size_t>(size_old, size_new) / 8ull);
 	std::mutex instructionMutex;
 	Threader threader;
 	constexpr size_t amount(4096);
@@ -186,7 +195,7 @@ bool BFT::DiffBuffers(char * buffer_old, const size_t & size_old, char * buffer_
 				// We only care about repeats larger than 36 bytes.
 				if (inst->newData.size() > 36ull) {
 					// Upper limit (mx and my) reduced by 36, since we only care about matches that exceed 36 bytes
-					size_t max = std::min(inst->newData.size(), inst->newData.size() - 37ull);
+					size_t max = std::min<size_t>(inst->newData.size(), inst->newData.size() - 37ull);
 					for (size_t x = 0ull; x < max; ++x) {
 						const auto & value_at_x = inst->newData[x];
 						if (inst->newData[x + 36ull] != value_at_x)
@@ -227,7 +236,7 @@ bool BFT::DiffBuffers(char * buffer_old, const size_t & size_old, char * buffer_
 							writeGuard.release();
 
 							x = ULLONG_MAX; // require overflow, because we want next itteration for x == 0
-							max = std::min(inst->newData.size(), inst->newData.size() - 37ull);
+							max = std::min<size_t>(inst->newData.size(), inst->newData.size() - 37ull);
 							break;
 						}
 						x = y - 1;

src/BufferTools.h

@@ -5,43 +5,52 @@
 
 /** Namespace to keep buffer-related operations grouped together. */
 namespace BFT {
-	/** Compresses a source buffer into an equal or smaller sized destination buffer.
+	/** Compresses a source buffer into an equal or smaller-sized destination buffer.
+	After compression, it applies a small header describing how large the uncompressed buffer is.
+	---------------
+	| buffer data |
+	---------------	
+	v
+	-----------------------------------------
+	| compression header | compressed data  |
+	-----------------------------------------
 	@param	sourceBuffer		the original buffer to read from.
-	@param	sourceSize			the size in bytes of the source buffer.
+	@param	sourceSize			the size of the source buffer in bytes.
 	@param	destinationBuffer	pointer to the destination buffer, which will hold compressed contents.
 	@param	destinationSize		reference updated with the size in bytes of the compressed destinationBuffer.
 	@return						true if compression success, false otherwise. */
 	bool CompressBuffer(char * sourceBuffer, const size_t & sourceSize, char ** destinationBuffer, size_t & destinationSize);
-	/** Decompressess a source buffer into an equal or larger sized destination buffer.
+	/** Decompressess a source buffer into an equal or larger-sized destination buffer.
 	@param	sourceBuffer		the original buffer to read from.
-	@param	sourceSize			the size in bytes of the source buffer.
+	@param	sourceSize			the size of the source buffer in bytes.
 	@param	destinationBuffer	pointer to the destination buffer, which will hold decompressed contents.
 	@param	destinationSize		reference updated with the size in bytes of the decompressed destinationBuffer.
 	@return						true if decompression success, false otherwise. */
 	bool DecompressBuffer(char * sourceBuffer, const size_t & sourceSize, char ** destinationBuffer, size_t & destinationSize);
-	/** Processes both input buffers, differentiating them, generating an output (compressed) diff-buffer, ready to be written to disk.
+	/** Processes two input buffers, diffing them.
+	Generates a compressed instruction set dictating how to get from the old buffer to the new buffer.
 	@note						caller expected to clean-up buffer_diff on their own
 	@param	buffer_old			the older of the 2 buffers.
 	@param	size_old			the size of the old buffer.
 	@param	buffer_new			the newer of the 2 buffers.
 	@param	size_new			the size of the new buffer.
 	@param	buffer_diff			pointer to store the diff buffer at.
 	@param	size_diff			reference updated with the size of the compressed diff buffer.
-	@param	instructionCount	optional pointer to update with the number of instructions processed.
+	@param	instructionCount	(optional) pointer to update with the number of instructions processed.
 	@return						true if diff success, false otherwise. */
 	bool DiffBuffers(char * buffer_old, const size_t & size_old, char * buffer_new, const size_t & size_new, char ** buffer_diff, size_t & size_diff, size_t * instructionCount = nullptr);
-	/** Uses a compressed diff buffer to patch a source buffer into an updated destination buffer
+	/** Reads from a compressed instruction set, uses it to patch the 'older' buffer into the 'newer' buffer
 	@note						caller expected to clean-up buffer_new on their own
 	@param	buffer_old			the older of the 2 buffers.
 	@param	size_old			the size of the old buffer.
 	@param	buffer_new			pointer to store the newer of the 2 buffers.
 	@param	size_new			reference updated with the size of the new buffer.
-	@param	buffer_diff			the compressed diff buffer.
+	@param	buffer_diff			the compressed diff buffer (instruction set).
 	@param	size_diff			the size of the compressed diff buffer.
-	@param	instructionCount	optional pointer to update with the number of instructions processed.
+	@param	instructionCount	(optional) pointer to update with the number of instructions processed.
 	@return						true if patch success, false otherwise. */
 	bool PatchBuffer(char * buffer_old, const size_t & size_old, char ** buffer_new, size_t & size_new, char * buffer_diff, const size_t & size_diff, size_t * instructionCount = nullptr);
-	/** Generate a hash value for the buffer provided. 
+	/** Generates a hash value for the buffer provided, using the buffers' contents.
 	@param	buffer				pointer to the buffer to hash.
 	@param	size				the size of the buffer.
 	@return						hash value for the buffer. */