Skip to content

Commit e41dc98

Browse files
committed
Merge branch 'release/1.0.0'
2 parents e4b972f + 602babd commit e41dc98

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

78 files changed

+2849
-2736
lines changed

.gitignore

+24-16
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,39 @@
1-
*.py[co]
1+
# Python Bytecode
2+
*.py[cod]
23

3-
# Packages
4-
*.egg
4+
# C Extensions
5+
*.so
6+
7+
# Packaging
58
*.egg-info
6-
dist
7-
build
8-
eggs
9-
parts
9+
.eggs
10+
.packaging
11+
12+
# Virtual Environment Pseudo-Chroot
1013
bin
14+
include
15+
lib
16+
share
17+
tmp
18+
usr
1119
var
12-
sdist
13-
develop-eggs
14-
.installed.cfg
1520

16-
# Installer logs
17-
pip-log.txt
18-
19-
# Unit test / coverage reports
21+
# Unit Test / Coverage Reports
22+
.cache
23+
.cagoule.db
2024
.coverage
2125
.tox
22-
.cache
26+
coverage.xml
2327
htmlcov
2428

2529
# Translations
2630
*.mo
31+
*.pot
2732

2833
# Mac
2934
.DS_Store
3035

31-
coverage.xml
36+
# Completion Integration
37+
.ropeproject
38+
tags
39+

.pre-commit-config.yaml

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
- repo: https://github.com/pre-commit/pre-commit-hooks.git
2+
sha: c8a1c91c762b8e24fdc5a33455ec10662f523328
3+
hooks:
4+
- id: check-added-large-files
5+
- id: check-ast
6+
- id: check-byte-order-marker
7+
- id: check-merge-conflict
8+
- id: check-symlinks
9+
- id: debug-statements
10+
- id: detect-private-key
11+
- id: end-of-file-fixer
12+
- id: check-json
13+
- id: check-xml
14+
- id: check-yaml

.travis.yml

+11-19
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
language: python
22
sudo: false
3+
cache: pip
34

45
addons:
56
apt:
@@ -10,35 +11,27 @@ addons:
1011
- mongodb-org-server
1112
- mongodb-org-shell
1213

13-
services:
14-
- mongodb
15-
1614
branches:
1715
except:
18-
- /^feature/.*$/
16+
- /^[^/]+/.+$/
1917

2018
python:
21-
- pypy
22-
- pypy3
2319
- "2.7"
20+
- "pypy"
21+
- "pypy3"
2422
- "3.3"
2523
- "3.4"
2624
- "3.5"
2725

28-
env:
29-
- PYTHONOPTIMIZE=
30-
- PYTHONOPTIMIZE=2
31-
32-
matrix:
33-
exclude:
34-
- python: pypy
35-
env: PYTHONOPTIMIZE=2
36-
- python: pypy3
37-
env: PYTHONOPTIMIZE=2
26+
install:
27+
- travis_retry pip install --upgrade setuptools pip pytest pytest-cov codecov 'setuptools_scm>=1.9'
28+
- pip install -e '.[development,logger]'
3829

39-
install: travis_retry .travis/install.sh
30+
script:
31+
python setup.py test
4032

41-
script: tox
33+
after_script:
34+
bash <(curl -s https://codecov.io/bash)
4235

4336
notifications:
4437
irc:
@@ -51,4 +44,3 @@ notifications:
5144
template:
5245
- "%{repository_slug}:%{branch}@%{commit} %{message}"
5346
- "Duration: %{duration} - Details: %{build_url}"
54-

.travis/install.sh

-21
This file was deleted.

.travis/run.sh

-8
This file was deleted.

Makefile

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
PROJECT = marrow.mongo
2+
USE = development,logger
3+
4+
.PHONY: all develop clean veryclean test release
5+
6+
all: clean develop test
7+
8+
develop: ${PROJECT}.egg-info/PKG-INFO
9+
10+
clean:
11+
find . -name __pycache__ -exec rm -rfv {} +
12+
find . -iname \*.pyc -exec rm -fv {} +
13+
find . -iname \*.pyo -exec rm -fv {} +
14+
rm -rvf build htmlcov
15+
16+
veryclean: clean
17+
rm -rvf *.egg-info .packaging
18+
19+
test: develop
20+
./setup.py test
21+
22+
release:
23+
./setup.py register sdist bdist_wheel upload ${RELEASE_OPTIONS}
24+
@echo -e "\nView online at: https://pypi.python.org/pypi/${PROJECT} or https://pypi.org/project/${PROJECT}/"
25+
@echo -e "Remember to make a release announcement and upload contents of .packaging/release/ folder as a Release on GitHub.\n"
26+
27+
${PROJECT}.egg-info/PKG-INFO: setup.py setup.cfg marrow/mongo/core/release.py
28+
@mkdir -p ${VIRTUAL_ENV}/lib/pip-cache
29+
pip install --cache-dir "${VIRTUAL_ENV}/lib/pip-cache" -Ue ".[${USE}]"
30+

README.rst

+136-16
Original file line numberDiff line numberDiff line change
@@ -18,17 +18,12 @@ Introduction
1818
============
1919

2020
Marrow Mongo is a collection of small, focused utilities written to enhance use of the `PyMongo native MongoDB driver
21-
<http://api.mongodb.com/python/current/>`_ without the overhead, glacial update cycle, complexity, and head-space
21+
<http://api.mongodb.com/python/current/>`__ without the overhead, glacial update cycle, complexity, and head-space
2222
requirements of a full active record object document mapper. Additionally, it provides a very light-weight database
23-
connection plugin for the `WebCore web framework <https://github.com/marrow/WebCore>`_ and Python standard logging
23+
connection plugin for the `WebCore web framework <https://github.com/marrow/WebCore>`__ and Python standard logging
2424
adapter to emit logs to MongoDB.
2525

2626

27-
Rationale and Goals
28-
-------------------
29-
30-
31-
3227
Installation
3328
============
3429

@@ -38,8 +33,8 @@ Installing ``marrow.mongo`` is easy, just execute the following in a terminal::
3833

3934
**Note:** We *strongly* recommend always using a container, virtualization, or sandboxing environment of some kind when
4035
developing using Python; installing things system-wide is yucky (for a variety of reasons) nine times out of ten. We
41-
prefer light-weight `virtualenv <https://virtualenv.pypa.io/en/latest/virtualenv.html>`_, others prefer solutions as
42-
robust as `Vagrant <http://www.vagrantup.com>`_.
36+
prefer light-weight `virtualenv <https://virtualenv.pypa.io/en/latest/virtualenv.html>`__, others prefer solutions as
37+
robust as `Vagrant <http://www.vagrantup.com>`__.
4338

4439
If you add ``marrow.mongo`` to the ``install_requires`` argument of the call to ``setup()`` in your application's
4540
``setup.py`` file, marrow.mongo will be automatically installed and made available when your own application or
@@ -56,11 +51,11 @@ Development Version
5651

5752
|developstatus| |developcover| |ghsince| |issuecount| |ghfork|
5853

59-
Development takes place on `GitHub <https://github.com/>`_ in the
60-
`marrow.mongo <https://github.com/marrow/mongo/>`_ project. Issue tracking, documentation, and downloads
54+
Development takes place on `GitHub <https://github.com/>`__ in the
55+
`marrow.mongo <https://github.com/marrow/mongo/>`__ project. Issue tracking, documentation, and downloads
6156
are provided there.
6257

63-
Installing the current development version requires `Git <http://git-scm.com/>`_, a distributed source code management
58+
Installing the current development version requires `Git <http://git-scm.com/>`__, a distributed source code management
6459
system. If you have Git you can run the following to download and *link* the development version into your Python
6560
runtime::
6661

@@ -73,12 +68,137 @@ You can then upgrade to the latest version at any time::
7368

7469
If you would like to make changes and contribute them back to the project, fork the GitHub project, make your changes,
7570
and submit a pull request. This process is beyond the scope of this documentation; for more information see
76-
`GitHub's documentation <http://help.github.com/>`_.
71+
`GitHub's documentation <http://help.github.com/>`__.
7772

7873

79-
Getting Started
80-
===============
74+
Documents
75+
=========
76+
77+
This package utilizes the `Marrow Schema <https://github.com/marrow/schema>`__ declarative schema toolkit and extends
78+
it to encompass MongoDB data storage concerns. You define data models by importing classes describing the various
79+
components of a collection, such as ``Document``, ``ObjectId``, or ``String``, then compose them into a declarative
80+
class model. For example, if you wanted to define a simple user account model, you would begin by importing::
81+
82+
from marrow.mongo import Index, Document
83+
from marrow.mongo.field import ObjectId, String, Number, Array
84+
85+
One must always import ``Document`` from ``marrow.mongo.core`` prior to any import of registered fields from
86+
``marrow.mongo``. As a note, due to the magical nature of this plugin import registry, it may change in future feature
87+
releases. The old interface will be deprecated with a warning for one feature version first, however; pin your
88+
dependencies.
89+
90+
91+
Defining Documents
92+
------------------
93+
94+
Now we can define our own ``Document`` subclass::
95+
96+
class Account(Document):
97+
username = String(required=True)
98+
name = String()
99+
locale = String(default='en-CA-u-tz-cator-cu-CAD', assign=True)
100+
age = Number()
101+
102+
id = ObjectId('_id', assign=True)
103+
tag = Array(String(), default=lambda: [], assign=True)
104+
105+
_username = Index('username', unique=True)
106+
107+
Broken down::
108+
109+
class Account(Document):
110+
111+
No surprises here, we subclass the Document class. This is required to utilize the metaclass that makes the
112+
declarative naming and order-presrving sequence generation work. We begin to define fields::
113+
114+
username = String(required=True)
115+
name = String()
116+
locale = String(default='en-CA-u-tz-cator-cu-CAD', assign=True)
117+
118+
Introduced here is ``required``, indicating that when generating the *validation document* for this document to
119+
ensure this field always has a value. This validation is not currently performed application-side. Also notable is the
120+
use of ``assign`` on a string field; this will assign the default value during instantiation. Then we have a different
121+
type of field::
122+
123+
age = Number()
124+
125+
This allows storage of any numeric value, either integer or floating point. Now there is the record identifier::
126+
127+
id = ObjectId('_id', assign=True)
128+
129+
Marrow Mongo does not assume your documents contain IDs; there is no separation internally between top-level documents
130+
and "embedded documents", leaving the declaration of an ID up to you. You might not always wish to use an ObjectID,
131+
either; please see MongoDB's documentation for discussion of general modelling practices. The first positional
132+
parameter for most non-complex fields is the name of the MongoDB-side field. Underscores imply an attribute is
133+
"protected" in Python, so we remap it by assigning it to just ``id``. The ``assign`` argument here ensures whenever a
134+
new ``Account`` is instantiated an ObjectID will be immediately generated and assigned.
135+
136+
Finally there is an array of tags::
137+
138+
tag = Array(String(), default=lambda: [], assign=True)
139+
140+
This combines what we've been using so far into one field. An ``Array`` is a *complex field* (a container) and as such
141+
the types of values allowed to be contained therein may be defined positionally. (If you want to override the field's
142+
database-side name, pass in a ``name`` as a keyword argument.) A default is defined as an anonymous callback function
143+
which constructs a new list on each request. The default will be executed and the result assigned automatically during
144+
initialization as per ``id`` or ``locale``.
145+
146+
Lastly we define a unique index on the username to speed up any queries involving that field::
147+
148+
_username = Index('username', unique=True)
149+
150+
151+
Instantiating Documents
152+
-----------------------
153+
154+
With a document schema defined we can now begin populating data::
155+
156+
alice = Account('amcgregor', "Alice Bevan-McGregor", age=27)
157+
print(alice.id) # Already has an ID.
158+
print(alice.id.generation_time) # This even includes the creation time.
159+
160+
As can be seen above construction accepts positional and keyword parameters. Fields will be filled, positionally, in
161+
the order they were defined, unless otherwise adjusted using the ``adjust_attribute_sequence`` decorator.
162+
163+
Assuming a ``pymongo`` collection is accessible by the variable name ``collection`` we can construct our index::
164+
165+
Account._username.create_index(collection)
166+
167+
There is no need to run this command more than once unless the collection is dropped.
168+
169+
Let's insert our record::
170+
171+
result = collection.insert_one(alice)
172+
assert result.acknowledged and result.inserted_id == alice.id
173+
174+
Yup, that's it. Instances of ``Document`` are directly usable in place of a dictionary argument to ``pymongo``
175+
methods. We then validate that the document we wanted inserted was, in fact, inserted. Using an assert in this way,
176+
this validation will not be run in production code run with the ``-O`` option passed (or ``PYTHONOPTIMIZE``
177+
environment variable set) in the invocation to Python.
178+
179+
180+
Querying Documents
181+
------------------
182+
183+
Now that we have a document stored in the database, let's retrieve it back out and get the result as an ``Account``
184+
instance::
185+
186+
record = collection.find_one(Account.username == 'amcgregor')
187+
record = Account.from_mongo(record)
188+
print(record.name) # Alice Bevan-McGregor
189+
190+
Several things are going on here. First it's important to note that Marrow Mongo isn't making the query happen for
191+
you, and does not automatically cast dictionaries to ``Document`` subclasses when querying. The first line
192+
demonstrates the native approach to building *filter documents*, the first argument to ``find`` or ``find_one``.
193+
194+
You can use standard Python comparison operators, bitwise operators, and several additional querying methods through
195+
class-level access to the defined fields. The result of one of these operations or method calls is a dictionary-like
196+
object that is the query. They may be combined through bitwise and (``&``) and bitwise or (``|``) operations, however
197+
due to Python's order of operations, individual field comparisons must be wrapped in parenthesis if combining.
81198

199+
Combining produces a new ``Ops`` instance, so it is possible to use these to pre-construct parts of queries prior to
200+
use. As a tip, it can save time (and visual clutter) to assign the document class to a short, single-character
201+
variable name to make repeated reference easier.
82202

83203

84204
Version History
@@ -153,7 +273,7 @@ OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
153273
:target: https://github.com/marrow/mongo/issues
154274
:alt: Github Issues
155275

156-
.. |ghsince| image:: https://img.shields.io/github/commits-since/marrow/mongo/1.0.svg
276+
.. |ghsince| image:: https://img.shields.io/github/commits-since/marrow/mongo/1.0.0.svg
157277
:target: https://github.com/marrow/mongo/commits/develop
158278
:alt: Changes since last release.
159279

0 commit comments

Comments
 (0)