Lift Framework 3.0-M8: Security and milestone release

This release features a dependency update to fix a security issue. See below.

It also features a large number of bug fixes and targeted improvements
based on feedback from actual use of Lift 3. It is expected to be the last Lift 3
milestone release, with the next build expected to be RC1. We don't expect
to land any more feature work before releasing Lift 3.0.0, and would like folks
to start hammering this release to find any lurking bugs that haven't yet been
caught.

Security

Lift 3.0 milestones were found to be using an outdated version of Apache
Commons FileUpload, which had an unpatched DoS vulnerability.

Lift 3.0-M8 bumps the dependency to the latest version, 1.3.1, which fixes
this vulnerability. Additionally, if you're using sbt, you can directly depend
on commons-fileupload 1.3.1 to evict the Lift dependency without upgrading
Lift. The versions are compatible with respect to Lift's usage.

Compatibility Notes

Lift 3.0-M7 was the last Lift build built with java 7. Future builds and the
final Lift 3.0.0 release will be built on Java 8. If you use Java 7, please
make sure to add the commons-fileupload 1.3.1 dependency mentioned
above so you are protected from the DoS vulnerability.

Also, earlier builds of Lift 3.0 defaulted the Content-Security-Policy to
report only in dev mode (which will produce error messages and logs, but
not prevent anything from actually loading/running), but actually enforced
the security policy in production. For the initial Lift 3.0.0 release, we are
relaxing this to allow for an easier transition: the default will be report-only
in both dev mode and production mode. Lift 3.1 will return to a secure
production default.

You can get the previous behavior by setting:

LiftRules.securityRules = () => SecurityRules(enforceInOtherModes = true)

You can also enforce in dev mode by passing the enforceInDevMode = true
parameter.

Note about M7

We cut M7 a little earlier than we wanted to announce another milestone,
so rather than announcing it we left the artifacts but made no official
announcement. The release notes below include the changes in M7.

Improvements

• (#1726) LiftRules.autoIncludeAjaxCalc, which decides whether or not to
  include Lift's JavaScript, now makes no attempt to include the lift.js file if the
  current request is stateless. Serving lift.js triggers session creation, so a
  page that was stateless for performance reasons but included lift.js would
  still cause session churn. The new default avoids this, though you can return
  to the old behavior if preferred.
• (#1710) Until now, render produced a Document, which is a now-deprecated Scala
  intermediary type. Passing that Document to pretty or compact would produce
  the desired kind of JSON output. We now drop Document from the pipeline,
  with compact(render) and pretty(render) producing deprecation warnings.
  These forms will be removed in 3.1, in favor of compactRender and prettyRender.
  The new serialization code is also both faster and more customizable, allowing
  rendering to any Java Appendable and some more customization on how
  renderings are formatted.
• (#1736) JSON serialization is well-specified, but when serializing for parsing
  by a JS language parser rather than a JSON parser (e.g., in JSON-P), escaping
  works a little differently. Lift now supports changing characters that are escaped
  and provides JS-specific convenience configurations for serializing JSON in
  ways that won't break for these cases.
• (#1741) RestHelper will now properly handle async responses wrapped in
  Scala Futures. Before, it only handled LAFuture. These async responses
  are handled using the same continuation mechanism that powers Lift's comets.
• (#1717) Deserializing JSON to a List[Option[T]] did not work. It now works
  correctly, deserializing null to None.
• (#1719) lift-json's write function can now take primitive values to serialize (e.g.,
  Serialization.write(3)). Before, it only supported object types.
• (#1752) Lift 3.0 finds on* attributes and other embedded JS and separates
  it out into actual JS that is invoked separately. This helps to enforce tighter
  Content-Security-Policy restrictions. Before, in AJAX responses that included
  HTML, we were not performing this event extraction. Now, event extraction
  happens for both pages and AJAX responses uniformly.

Fixes

• (#1711) S.appendJs could behave strangely when invoked from within
  JsCmd construction, and particularly in Wiring and in certain comet contexts.
  It should now behave predictably across the board.
• (#1714) MatchWithoutCurrentValue did not work correctly for Menus
  that did not include a *. This is now fixed.
• (#1718) Deserializing an incorrect type to an Option[T] would produce a
  None. Now, only a null produces a None, something that can deserialize
  to a T produces a Some[T], and a non-null value that cannot deserialize
  to a T produces a MappingException.
• (#1721) Lift 3's default client-side logError implementation was broken and
  caused a stack overflow. It is now fixed.
• (#1733) lift-markdown was failing to correctly break up parsing by newline.
  This was only on Java 8 due to a change in regex behavior in Java; it is now
  fixed.
• (#1734) CssUrlPrefixer, which prepends the context path of the application
  to served CSS, was failing to properly parse certain URLs, particularly those
  that included , and %. It was also always emitting URLs wrapped in single
  quotes irrespective of the source CSS's style. All of these issues are now fixed.
• (#1737) Msgs now appends any JS it needs to the page JS, rather than
  inserting it into the page markup.
• (#1744) Fixes for IE8 in lift.js. We were relying on some functionality that
  does not exist in IE8. We now have a polyfill for this functionality.
• (#1748) Lift's non-jQuery implementation of internal functionality now properly
  sets X-Requested-With for AJAX requests, thus allowing Lift to internally
  recognize AJAX requests (via Req's xhr_? method).
• (#1749) Our mongo dependencies were upgraded to the latest driver that
  still provides support for some of our older customization strategies. The
  older customization is now deprecated for removal in 3.1, so that we can
  catch up to the latest mongo driver. Note that the driver we currently depend
  on is still compatible with the latest server versions.
• (#1750) When trying to use Menu.item, we were trying to identify the current
  page using S.request. This failed when we were trying to build menu HTML
  from an AJAX callback, since S.request no longer referred to the current page,
  but rather the AJAX request. We now use S.originalRequest, which does refer
  to the current page even in an AJAX callback.

Deprecations

• (#1710) In lift-json, compact and pretty (previously used in conjunction with
  render to serialize JSON to a String) are now deprecated in favor of compactRender
  and prettyRender. They will be removed in Lift 3.1.
• (#1742) FocusOnLoad is now deprecated in favor of S.appendJs(Focus(...))
  and/or using the HTML5 autofocus attribute.
• (#1749) Our mongo dependencies were upgraded to the latest driver that
  supports some of our older customization strategies. The older customization
  is now deprecated for removal in 3.1. In particular, these are defineDbAuth
  and useSession wrappers.