Skip to content

Packaging Rock Core Updates

Jump to bottom Edit New page
Nick Airdo edited this page Feb 9, 2022 · 68 revisions

RockUpdate v2.0

This is intended for internal use. This page explains how official Rock-Update packages should be made and the corresponding design notes.

2.0 Packaging Design Overview

An official Rock update (e.g., v12.5 or 1.12.5.0) will consist of one zipped *.rockpkg package file that includes any changed file items for that update/release/patch and a file that contains the list of items to be deleted for that release.

Structure The package will consist of a \install\ folder and a \content\ folder:

+---content
|   |   web.config.rock.xdt
|   |
|   +---bin
|   \---Blocks
\---install
        deletefile.lst

The content folder will contain all the updated items from the 'RockWeb' project. The install folder contains the deletefile.lst if one exists for that update. The deletefile.lst contains the files to be removed for that update.

Upgrading Rock

As of v13.0 (6f8f8be) the new RockUpdate block no longer uses NuGet. Instead it is based on the approach used by the RockShop package installation and uses a simple unzip approach to updating.

v13.1 - Roslyn

If someone upgrades Rock rapidly from one version to another, due to the fact that Roslyn might be compiling the previously installed *.ascx.cs blocks we have seen the RockUpdate fail to unzip the Roslyn compiler:

There is a problem installing v1.13.0. We were not able to replace an important file (The process cannot access the file 'C:\inetpub\wwwroot\Bin\roslyn\VBCSCompiler.exe' because it is being used by another process.) after the update.

To address this, a change (8390c2d) was made in v13.1 to disallow updating Rock if Roslyn is still compiling.

Additional Details

For the moment, additional details are documented internally but they can be moved here at a later date.

The remainder of this is for the old v1 updater.

RockUpdate v1.0

This is intended for internal use. This page explains how official Rock-Update packages should be made and it also includes design notes and instructions regarding exporting pages, blocks, etc. from Rock. This is for Trello card #34 and Trello card #133.

A new RockUpdate v2.0 is being completed which removes the NuGet portion of the process so we can fetch the update bits from our own cloud storage.

Forward: When Creating a New 1.x Release

  • we'll branch develop to release-X.Y
  • we'll create a hotfix-X.Y.Z branch
  • any bugs we find MUST be fixed in the hotfix-X.Y.Z branch and should be merged back to the develop branch

Packaging Design Overview

An official Rock update (v1.0.3 for example) will consist of two NuGet packages. One will have "Rock" as the Id, and the other will have a unique Id (such as RockUpdate-1-0-3). The Rock package will simply have a dependency on the RockUpdate-* package, and the RockUpdate-* package contents will include any changed file items for that update/release/patch.

Why, you ask? Why not just put all those items into one single Rock package? Due to the way NuGet works, each Rock package would have to include every Rock file. When NuGet installs an updated package, it removes all the previous package file items and adds all the new package file items. We want small delta patches of only the files that changed since the last release.

With our approach, each release of the Rock package will have a dependency on the single, latest RockUpdate-* package and then that package will include a dependency on the immediate/prior RockUpdate-* package (recursively, all the way back to the first update. You can see these relationships depicted in the following diagram:

As time goes on, a Rock installation will be comprised of a single Rock package and all the delta/patch packages (RockUpdate-1-0-0, RockUpdate-1-0-1, RockUpdate-1-0-2, etc.) that lead up to that release. Once installed the packages will be found inside the App_Data\packages folder (as defined in the WebProjectManager classes GetWebRepositoryDirectory() method.

So, the NuGet package metadata for a couple of updates might look something like this:

[Caption][img1] [img1]: Attachments/Exporting-and-Packaging/rock-package-metadata.png

Without exception, a minor version (1.X.0) update will require that it be installed before any patches can be installed. This is enforced in the patch release (1.X.Y) by using the "Tags" field in the package specifying "requires-1.X.0".

Super Patches

Additionally, as long as a patch was created with the RockPackageBuilder specifying the -l option with the version number of the previous minor release (1.X.0), the patch package will include all changes since that minor release. This is a super-patch. In this case, the "Tags" field in that package can simply specify "requires-1.X.0" and doing so will allow the Rock admin to upgrade directly to that patch after the minor release is installed -- bypassing all other patches between the minor release and the latest patch release.

The RockUpdate block handles the update process.

Regarding Uninstall

Because Rock packages may occasionally contain embedded migrations inside the Rock.dll, it would be quite difficult to support a back-out or restore-to-previous-version type of system.

Major, Minor, Patch Release

As mentioned in the semantic versioning specification, "patches" and "minor" releases should always be backward compatible. A "major" release should only occur when Rock runs into the situation where backwards compatibility cannot be maintained. Because of this, we need to put safe guards into the RockUpdate block to prevent a user simply from attempting to do a 'simple update' to a greater major version number (for example, from version 1.x.x to version 2.x.x or greater).

There should be no problems using this RockUpdate block approach to upgrade any minor or patch release of Rock.

The official "product version fullname" of Rock is noted inside the AssemblyInfo.cs. This is the friendly name of the release/patch that the user sees on certain pages and can be obtained by calling the Rock.Version's GetRockProductVersionFullName() method.

    [assembly: AssemblyInformationalVersion( "Rock Humphreys 0.1.0 (alpha)" )]

To get only the official version number call the Rock.Version's GetRockProductVersionNumber() method.

Rock Package MetaData

  • Id: - This must always be "Rock"
  • Version: - This is the semantic version of the release.
  • Title: - This should be a friendly name for the release (if any; otherwise Rock Update will suffice).
  • Summary/Description: - This will be displayed to the user so they can understand more about the release.
  • Tags: - Use the format "requires-A.B.C" to denote that the A.B.C Rock package MUST be installed prior to this package installation.
  • Release notes: - These will be displayed immediately following the install (if successful).
  • Dependencies: - This should refer to the corresponding RockUpdate* package.

RockUpdate Package MetaData & Package Contents

  • Id: - This should be something that starts with "RockUpdate". The Id for each update should be unique unless you intend on it completely replacing the previous update.
  • Version: - This is largely unused at this point, but it may make sense to simply use the same versioning as the corresponding Rock package.
  • Title: - unused.
  • Package contents: -
    • the Rock.Version.DLL should always be included in the lib folder along with any other assemblies required with the release/patch.
    • All other content should be included under the NuGet content folder.
    • Any files that are to be deleted must be listed in a App_Data\deletefile.lst file. After installing, the RockUpdate block will remove each file listed in that file and then remove the deletefile.lst.
    • Any XML files (such as foo\example.config) that are to be transformed must have a corresponding *.rock.xdt file (as in foo\example.config.rock.xdt). Details regarding the Transformation Syntax can be found on the MSDN site. (There is also a testing tool over at appHarbor) After installing, the system will perform the transform and then remove the *.rock.xdt files.

Steps for Creating a Rock Update (release)

IMPORTANT! You must configure your git client to fetch updated tags. Otherwise it's possible that you will not fetch items that were re-tagged since the initial build (perhaps during the alpha release) when the tags were first pulled. You probably want to add this to your .git\config:

[remote "origin"]
...
  fetch = +refs/tags/*:refs/tags/*

The RockPackageBuilder will do most of the work when it comes to packaging. It will get list of all files added/changed/deleted since last tag, build a new nuget package (Id: RockUpdate-X-Y-Z) with a dependency to the last RockUpdate, and build a Rock package (Id: Rock) with a dependency to the current RockUpdate-X-Y-Z package. However, there are still some steps that humans must perform.

These are the steps we'll follow when creating an official update:

  1. Merge-Master Role (full release)

Moved to our internal site over at http://admin.sparkdevnetwork.com/page/592

  1. Merge-Master Role (hotfix)

Moved to our internal site over at http://admin.sparkdevnetwork.com/page/592

  1. Package-Master Role

Moved to our internal site over at http://admin.sparkdevnetwork.com/page/592

  1. Package-Master Role
    1. Merge the develop (or other appropriate branch) into the master branch (causes issues to auto-close and email the issue reporters)
      1. Switch to the master branch
      2. Select Merge (verify that the latest release is selected) and press Create Merge-Commit
      3. You should be prompted to just "Fast-Forward" (if not somethings not right), press Fast Forward
      4. The changes have been committed so now just Push.

RockUpdater Design Overview

Using the Rock.Services.NuGet.WebProjectManager class, the RockUpdate block simply lists any pending updates and displays the results of an update. All the real work to perform an update is done by the Rock.Services.NuGet.WebProjectManager (and related WebProjectSystem and RockProjectManager classes). The RockUpdate block uses the UpdateServerUrl Global Attribute when constructing a WebProjectManager to determine the location of the NuGet package repository.

RockUpdate

From a high level, when an update is performed by the RockUpdate block, it will:

  1. Write an app offline page.
  2. Use the WebProjectManager to perform an update.
  3. If successful:
    1. Move any new files found in the App_Data\<version> folder to their respective folders.
    2. Record the new version to the "RockInstanceId" System Setting.
    3. Register any new REST controllers.
    4. Send statistics to the www.rockrms.com server (if successful).
  4. if unsuccessful:
    1. Display the errors.
  5. Remove the app offline page.

The RockUpdate block also removes any old *.rdelete files found under app root every time it loads.

WebProjectManager

From a high level, when the WebProjectManager UpdatePackageAndBackup method is called, it will:

  1. Backup the old package file (Rock.x.y.z.nupkg) to App_Data\PackageRestore.
  2. Use the RockProjectManager to invoke the NuGet UpdatePackageReference method. NuGet will then invokes the RockProjectManager's ExtractPackageFilesToProject method which:
    1. Extracts any *.rock.xdt files into a respective subfolder in the App_Data\PackageRestore\xdt folder
    2. Runs the transform against the intended target and copies the output to the respective App_Data\PackageRestore\xdtfolder.
    3. Calls the NuGet base ExtractPackageFilesToProject method to continue the update which invokes the WebProjectSystem's AddFile method for each new file that is being added. In AddFile:
      1. If file is a DLL that is not in the bin folder, it moves the existing one to a *.rdelete file.
      2. Calls the NuGet base AddFile method if the file is not a *.rock.xdt file.
      3. If file is the App_Data\deletefile.lst, performs the delete of all files referenced.
    4. Moves the transform files to the respective folder under app root.
    5. Deletes the App_Data\PackageRestore folder.
  3. If exceptions:
    1. Restore the old package file back to the App_Data\Packages folder.

Building Rockit SDK

When a new release of the Rockit SDK is built, the proper Rock and RockUpdate nuget packages must be put into the App_Data/packages folder, otherwise when the SDK is updated, it could pull down every Rock version since the beginning and extract all those files. That will not end well.

For example, lets say you are building a Rockit SDK for version 1.7.2. Then the App_Data/packages folder must have the following STUB nuget files (they don't have to include any of the actual updated code/DLLs):

  • Rock.1.7.2.nupkg
  • RockUpdate-1-7-2.1.7.2.nupkg
  • RockUpdate-1-7-1.1.7.1.nupkg
  • RockUpdate-1-7-0.1.7.0.nupkg

Why are all those needed? Because when the user updates to Rock v.1.7.3, the RockUpdate 1.7.3 package will be pulled down AND it probably has a dependency on RockUpdate v1.7.0. If that v1.7.0 package was not on the filesystem, nuget would go fetch RockUpdate 1.7.0 -- and it probably has a dependency on RockUpdate 1.6.10, and you see where this goes. That will continue until the end of the dependency chain is reached. That's why we want to include the oldest possible RockUpdate package that any future Rock updates will depend on.

Notes

The RockInstanceId key (public static readonly string ROCK_INSTANCE_ID = "RockInstanceId";) is used by the SystemSettings.GetRockInstanceId() method to retrieve a globally unique identifier for each installation of Rock. The value comes from the Guid of the RockInstanceId Attribute:

   SELECT [Guid]
   FROM [Attribute]
   WHERE [Key] = 'RockInstanceId'

Merging Hotfix to Develop

Once you're finished with a hotfix branch, finish it by doing the following:

  1. Switch to develop branch.
  2. Click Merge.
  3. Select the hotfix commit and select Create Merge-Commit.
  4. Create a single new regular migration for each of the plugin hotfix migrations:
    1. Add-Migration MIGRATIONNAME
    2. Add the SQL to a file called timestamp_MIGRATIONNAME.sql in the Migrations folder
      1. Open the RockMigrationSQL.resx file and drag-drop the timestamp_MIGRATIONNAME.sql, then save it.
    3. Add a comment before each section referencing the plugin hotfix migration number.
    4. Comment out the hotfix migration and include a reference to the regular migration.
    5. TEST

Code-Gen Operation

TBD

Update / Appendix

Aug 9, 2021

Alpha/Beta Repo - Due to increasing costs of our MyGet repo, today we removed the available RockUpdate-x.y.z packages that exist prior to v1.10.0 (RockUpdate-1-10-0.1.10.0.nupkg) What that means is, that 10.0 package will NOT include a dependency section so that it will no longer attempt to fetch anything missing prior v10.0. In other words, installing anything earlier than Rock v10 will no longer be supported in an alpha/beta environment.

Mar 11, 2021

Alpha/Beta Repo - Due to increasing costs of our MyGet repo, today we removed the available RockUpdate-x.y.z packages that exist prior to v1.9.0 (RockUpdate-1-9-0.1.9.0.nupkg) What that means is, that 9.0 package will NOT include a dependency section so that it will no longer attempt to fetch anything missing prior v9.0. In other words, installing anything earlier than Rock v9 will no longer be supported in an alpha/beta environment.

Oct 30, 2020

The RockUpdate-1-6-0.1.6.0.nupkg package in the regular (prod) repo was changed (again) to not have a dependency on the RockUpdate-1-5-5.1.5.5.nupkg.

July 1, 2020

Due to increasing costs of our MyGet repo, in the next week or so we're going to trim the available RockUpdate-x.y.z packages that exist prior to v1.8.0 (RockUpdate-1-8-0.1.8.0.nupkg). What that means is, that package will NOT include a dependency section so that it will no longer attempt to fetch anything missing prior to that update. In other words, installing anything earlier than Rock v8 will no longer be supported.

Oct 9, 2019

Due to increasing costs of our MyGet repo, in the next week or so we're going to trim the available RockUpdate-x.y.z packages that exist prior to v1.6.0 (RockUpdate-1-6-0.1.6.0.nupkg). What that means is, that package will NOT include a dependency section so that it will no longer attempt to fetch anything missing prior to that update. In other words, installing anything earlier than Rock v6 will no longer be supported.

Add a custom footer
Add a custom sidebar
Clone this wiki locally