Welcome to Ming Documentation

Ming is an Object Document Mapper (like an ORM but for Document based databases), for MongoDB. Ming extends pymongo providing:

• Declarative Models
• Schema Validation and Conversion
• Schema Evolution
• Pure InMemory MongoDB Implementation
• Unit of Work
• Identity Map
• One-To-Many, Many-To-One and Many-To-Many Relations

Getting Started

To get started with Ming just install it with:

$ pip install ming

Connecting to MongoDB

Before we start, make sure you have a copy of MongoDB running. First thing needed to start using Ming is to tell it how to connect to our instance of mongod. For this we use the create_datastore() function, this function creates a connection to the MongoDB instance, replicaset or cluster specified by the given URL:

from ming import create_datastore
from ming.odm import ThreadLocalODMSession

session = ThreadLocalODMSession(
    bind=create_datastore('mongodb://myuser:mypassword@localhost:27017/odm_welcome')
)

The ThreadLocalODMSession is the object all your models will use to interact with MongoDB and can be directly used to perform low-level mongodb oprations. While this provides no particular benefit over using pymongo directly it already permits to create and query documents:

>>> session.db.wiki_page.insert({'title': 'FirstPage',
...                              'text': 'This is my first page'})
[ObjectId('6413575b1cb6f0fe17cb480b')]
>>> session.db.wiki_page.find_one({'title': 'FirstPage'})
{'_id': ObjectId('6413575b1cb6f0fe17cb480b'), 'title': 'FirstPage', 'text': 'This is my first page'}

Using Models

Now that we know how to connect to the Database we can declare models which will be persisted on the database their session is associated to:

from ming import schema
from ming.odm import FieldProperty
from ming.odm.declarative import MappedClass

class WikiPage(MappedClass):
    class __mongometa__:
        session = session
        name = 'wiki_page'

    _id = FieldProperty(schema.ObjectId)
    title = FieldProperty(schema.String(required=True))
    text = FieldProperty(schema.String(if_missing=''))

Models can be created by creating them and flushing their changes to the database. A Model can then be queried back with the Model.query.find() method:

>>> # Creating a Document is enough to register it into the UnitOfWork
>>> WikiPage(title='FirstPage',
...          text='This is my first page')
<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb480c')
  title='FirstPage' text='This is my first page'>
>>> # Flush the unit of work to save changes on DB
>>> session.flush()
>>> 
>>> wp = WikiPage.query.find({'title': 'FirstPage'}).first()
>>> wp
<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb480b')
  title='FirstPage' text='This is my first page'>

To start working with Ming continue with the Ming ODM User Guide

Community

To get help with using Ming, use the Ming Users mailing list or the TurboGears Users mailing list.

Contributing

Yes please! We are always looking for contributions, additions and improvements.

The source is available on GitHub and contributions are always encouraged. Contributions can be as simple as minor tweaks to this documentation or to ming itself.

To contribute, fork the project and send a pull request.

Changes

See the Ming News / Release Notes for a full list of changes to Ming

Documentation Content

Ming ODM User Guide

Introduction

Ming provides an high-level abstraction for modeling objects in MongoDB. This higher-layer abstraction is referred in this document as the ODM since it is implemented in the “spirit” of object-relational mappers such as SQLAlchemy.

The ODM provides several features beyond those provided by basic PyMongo:

Schema Validation

Because MongoDB is schema-less, there are no guarantees given to the database client of the format of the data that may be returned from a query; you can put any kind of document into a collection that you want. The goal of Ming is to allow you to specify the schema for your data in Python code and then develop in confidence, knowing the format of data you get from a query.

UnitOfWork

Using ODM-enabled sessions allows you to operate on in-memory objects exclusively until you are ready to “flush” all changes to the database. Although MongoDB doesn’t provide transactions, the unit-of-work (UOW) pattern provides some of the benefits of transactions by delaying writes until you are fairly certain nothing is going to go wrong.

IdentityMap

In base Ming, each query returns unique document objects even if those document objects refer to the same document in the database. The identity map in Ming ensures that when two queries each return results that correspond to the same document in the database, the queries will return the same Python object as well.

Relations

Although MongoDB is non-relational, it is still useful to represent relationships between documents in the database. The ODM layer in Ming provides the ability to model relationships between documents as straightforward properties on Python objects.

Installing

To begin working with Ming, you’ll need to download and install a copy of MongoDB. You can find download instructions at http://www.mongodb.org/downloads

In order to install ming, as it’s available on PyPI, you can use pip (We recommend using a virtualenv for development.):

$ pip install ming

Alternatively, if you don’t have pip installed, download it from PyPi and run

$ python setup.py install

Note

To use the bleeding-edge version of Ming, you can get the source from GitHub and install it as above.

Connecting to MongoDB

Ming manages your connection to the MongoDB database using an object known as a DataStore. The DataStore is actually just a thin wrapper around a pymongo connection and Database object. (The actual Database object can always be accessed via the DataStore.db property of the DataStore instance).

>>> from ming import create_datastore
>>>
>>> datastore = create_datastore('mongodb://localhost:27017/tutorial')
>>> datastore
<DataStore None>
>>> # The connection is actually performed lazily
>>> # the first time db is accessed
>>> datastore.db
Database(MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True), 'tutorial')
>>> datastore
<DataStore Database(MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True), 'tutorial')>

Note

Ming also provides a “mongo in memory” implementation, which is non-persistent, in Python, and possibly much faster than MongoDB for very small data sets, as you might use in testing. To use it, just change the connection url to mim://

The ODM Session

Ming, like many object-relational mappers (ORMs), revolves around the idea of model classes. In order to create these classes, we need a way of connecting them to the datastore. Ming uses an object known as a Session to do this.

The session is responsible for this as well as maintaining the unit of work, identity map, and relations between objects. The ODM session itself is not designed to be thread-safe, so Ming provides a thread-local version of the session for safe operation in a multithreaded environment.

Usually you will rely on the session for everything and won’t use the datastore directly, you will just pass it to the Session and rely on the session itself:

>>> from ming import create_datastore
>>> from ming.odm import ThreadLocalODMSession
>>> 
>>> session = ThreadLocalODMSession(
...     bind=create_datastore('mim:///tutorial')
... )
>>> session
TLProxy of <session>
  <UnitOfWork>
    <new>
    <clean>
    <dirty>
    <deleted>
  <imap (0)>
>>> # The database and datastore are still available
>>> # through the session as .db and .bind
>>> session.db
mim.Database(tutorial)
>>> session.bind
<DataStore mim.Database(tutorial)>

Note

If you are using the TurboGears2 web framework the framework sets up a DataStore for you and passes it to your model.init_model function so there is no need to manually create the datastore.

ReplicaSets

Connecting to a ReplicaSet is possible by passing multiple hosts separated by comma to the DataStore and the replicaSet= option as the url:

from ming import create_datastore
from ming.odm import ThreadLocalODMSession

session = ThreadLocalODMSession(
    bind=create_datastore('mongodb://localhost:27017,localhost:27018/?replicaSet=foo')
)

When connecting to a ReplicaSet some useful options are available that define how Ming should behave when writing to the database. Additionally to replicaSet option you might want to check:

• w=X Which states how many members of the replica set should have replicated the data before and insertion/update is considered done. X can be a number or majority for more then half. See MongoDB Write Concern Option for additional details.
• readPreference=[primary|primaryPreferred|secondary|secondaryPreferred|nearest] which states if queries should only happen on the replicaset master of if they can be performed on secondary nodes.

See MongoDB Documentation for a full list of available options inside the mongo url.

Authentication

Connecting to a MongoDB instance or cluster using authentication can be done by passing the username and password values in the url itself:

from ming import create_datastore
from ming.odm import ThreadLocalODMSession

session = ThreadLocalODMSession(
    bind=create_datastore('mongodb://myuser:mypassword@localhost:27017/dbname')
)

Mapped Classes and Documents

In MongoDB documents are just plain dictionaries. In Ming ODM layer documents are represented by mapped classes, which inherit from ming.odm.mapped_class.MappedClass. MappedClasses do not descend from dict, to emphasize the difference between a mapped class (which may contain relations to other objects) and a MongoDB document (which may not).

To start working with MappedClasses you need a few more imports:

from ming import schema
from ming.odm import MappedClass
from ming.odm import FieldProperty, ForeignIdProperty

Mapped Classes also define the schema of your data, it means that whenever you are storing data into the MappedClass it gets validated against the schema. The attributes available to the document are declared as FieldProperty instances and their type for validation is specified in a declarative manner:

class WikiPage(MappedClass):
    class __mongometa__:
        session = session
        name = 'wiki_page'

    _id = FieldProperty(schema.ObjectId)
    title = FieldProperty(schema.String(required=True))
    text = FieldProperty(schema.String(if_missing=''))

The schema instances passed to FieldProperty also provide some options to ensure that they are not empty or to provide a default value when they are. See FancySchemaItem class for a list of available options.

Note

For a full list of available schema types refer to ming.schema module.

At the end of the model file, you should call Mapper.compile_all() to ensure that Ming has full information on all mapped classes:

from ming.odm import Mapper
Mapper.compile_all()

Type Annotations

Some type annotations are in Ming, but you need to add a hint to each class to help. The primary goal so far is to improve IDE experience. They may or may not work with mypy. Add some imports and the query: line to your models like this:

import typing

if typing.TYPE_CHECKING:
    from ming.odm.mapper import Query

...

class WikiPage(MappedClass):
    class __mongometa__:
        session = session
        name = 'wiki_page'

    query: 'Query[WikiPage]'

    ...

Creating Objects

Once we have the boilerplate out of the way, we can create instances of the WikiPage as any other Python class. One thing to notice is that we don’t explicitly call the save() method on the WikiPage; that will be called for us automatically when we flush() the session:

>>> wp = WikiPage(title='FirstPage',
...               text='This is a page')
>>> wp
<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
  title='FirstPage' text='This is a page'>

The previous line will just create a new WikiPage and store it inside the ODM UnitOfWork and IdentityMap in new state:

>>> session
TLProxy of <session>
  <UnitOfWork>
    <new>
      <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
          title='FirstPage' text='This is a page'>
    <clean>
    <dirty>
    <deleted>
  <imap (1)>
    WikiPage : 6413575b1cb6f0fe17cb481b => <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
        title='FirstPage' text='This is a page'>

As soon as we flush our session the Unit of Work is processed, all the changes will be reflected on the database itself and the object switches to clean state:

>>> session.flush()
>>> session
TLProxy of <session>
  <UnitOfWork>
    <new>
    <clean>
      <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
          title='FirstPage' text='This is a page'>
    <dirty>
    <deleted>
  <imap (1)>
    WikiPage : 6413575b1cb6f0fe17cb481b => <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
        title='FirstPage' text='This is a page'>

By default the session will keep track of the objects that has already seen and that are currently in clean state (which means that they have not been modified since the last flush to the session). If you want to trash away those objects from the session itself you can call the clear method

>>> session.clear()
>>> session
TLProxy of <session>
  <UnitOfWork>
    <new>
    <clean>
    <dirty>
    <deleted>
  <imap (0)>

Clearing the session gives you a brand new session, so keep in mind that after clearing it, it won’t have track anymore of the previous items that were created. While it is possible to flush the session multiple times, it is common practice in web applications to clear it only once at the end of the request.

Note

Both flushing the session and clearing it at the end of every request in web application can be provided automatically by the MingMiddleware WSGI Middleware.

Note

In case you are using the TurboGears2 Web Framework there is no need to use the MingMiddleware as TurboGears will automatically flush and clear the session for your at the end of each request.

Querying Objects

Once we have a WikiPage in the database, we can retrieve it using the .query attribute. The query attribute is a proxy to the Session query features which expose three methods that make possible to query objects _ClassQuery.get(), ODMSession.find() and ODMSession.find_and_modify():

>>> wp = WikiPage.query.get(title='FirstPage')
>>> 
>>> # Note IdentityMap keeps only one copy of the object when they are the same
>>> wp2 = WikiPage.query.find({'text': 'This is a page'}).first()
>>> wp is wp2
True

As you probably noticed while we directly retrieved the object when calling .get() we needed to append the .first() call to the result of .find(). That is because ODMSession.find() actually returns an ODMCursor which allows to retrieve multiple results, just get the first or count them:

>>> # Create a new page to see find in action
>>> wp2 = WikiPage(title='SecondPage', text='This is a page')
>>> session.flush()
>>> 
>>> WikiPage.query.find({'text': 'This is a page'}).count()
2
>>> WikiPage.query.find({'text': 'This is a page'}).first()
<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
  title='FirstPage' text='This is a page'>
>>> WikiPage.query.find({'text': 'This is a page'}).all()
[<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
  title='FirstPage' text='This is a page'>, <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481c')
  title='SecondPage' text='This is a page'>]

ODMSession.find() uses the MongoDB query language directly, so if you are already familiar with pymongo the same applies. Otherwise you can find the complete query language specification on MongoDB Query Operators documentation.

The find operator also allows to retrieve only some fields of the document, thus avoiding the cost of validating them all. This is especially useful to speed up queries in case of big subdocuments or list of subdocuments:

>>> WikiPage.query.find({'text': re.compile(r'^This')}, projection=('title',)).all()
[<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
  title='FirstPage' text=''>, <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481c')
  title='SecondPage' text=''>]

Note

Keep in mind that properties validation still applies, so when using the fields option you will still get the if_missing value if they declare one and the query will fail if the fields where required.

Notice that querying again a document that was retrieved with a fields limit won’t add the additional fields unless the refresh option is used:

>>> WikiPage.query.find({}).first()
<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
  title='FirstPage' text=''>
>>> WikiPage.query.find({}, refresh=True).first()
<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
  title='FirstPage' text='This is a page'>

Also applying the fields limit to a document already in the IdentityMap won’t do anything unless refresh is used (which is usually not what you want as you already paid the cost of validating the attributes, so there is not much benefit in throwing them away):

>>> docs = WikiPage.query.find({'text': re.compile(r'^This')}, projection=('title',)).all()
>>> docs[0].title, docs[0].text
('FirstPage', 'This is a page')
>>> docs[1].title, docs[1].text
('SecondPage', '')

Editing Objects

We already know how to create and get back documents, but an ODM won’t be of much use if it didn’t enable editing them. As Ming exposes MongoDB documents as objects, to update a MongoDB object you can simply change one of its properties and the UnitOfWork will track that the object needs to be updated:

>>> wp = WikiPage.query.get(title='FirstPage')
>>> session
TLProxy of <session>
  <UnitOfWork>
    <new>
    <clean>
      <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
          title='FirstPage' text='This is a page'>
    <dirty>
    <deleted>
  <imap (1)>
    WikiPage : 6413575b1cb6f0fe17cb481b => <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
        title='FirstPage' text='This is a page'>
>>> 
>>> wp.title = 'MyFirstPage'
>>> # Notice that the object has been marked dirty
>>> session
TLProxy of <session>
  <UnitOfWork>
    <new>
    <clean>
    <dirty>
      <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
          title='MyFirstPage' text='This is a page'>
    <deleted>
  <imap (1)>
    WikiPage : 6413575b1cb6f0fe17cb481b => <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
        title='MyFirstPage' text='This is a page'>
>>> 
>>> # Flush the session to actually apply the changes
>>> session.flush()

Another option to edit an object is to actually rely on ODMSession.find_and_modify() method which will query the object and update it atomically:

>>> WikiPage.query.find_and_modify({'title': 'MyFirstPage'},
...                                update={'$set': {'text': 'This is my first page'}},
...                                new=True)
<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb481b')
  title='MyFirstPage' text='This is my first page'>

This is often used to increment counters or acquire global locks in mongodb

Note

find_and_modify always refreshes the object in the IdentityMap, so the object in your IdentityMap will always get replaced with the newly retrieved value. Make sure you properly flushed any previous change to the object and use the new option to avoid retrieving a stale version of the object if you plan to modify it.

Ming also provides a way to update objects outside the ODM through ODMSession.update(). This might be handy when you want to update the object but don’t need the object back or don’t want to pay the cost of retrieving and validating it:

>>> WikiPage.query.update({'_id': wp._id}, {'$set': {'text': 'This is my first page!!!'}})
{'connectionId': None, 'updatedExisting': True, 'err': None, 'ok': 1.0, 'n': 1, 'nModified': 1}
>>> 
>>> # Update doesn't fetch back the document, so
>>> # we still have the old value in the IdentityMap
>>> WikiPage.query.get(wp._id).text
'This is my first page'
>>> 
>>> # Unless we refresh it.
>>> wp = session.refresh(wp)
>>> wp.text
'This is my first page!!!'

Note

The ODMSession.refresh() method provides a convenient way to refresh state of objects from the database, this is like querying them back with .find() using the refresh option. So keep in mind that any change not yet persisted on the database will be lost.

Deleting Objects

Deleting objects can be performed by calling the _InstQuery.delete() method which is directly exposed on objects:

>>> wp = WikiPage.query.get(title='MyFirstPage')
>>> wp.delete()
>>> 
>>> # We flush the session to actually delete the object
>>> session.flush()
>>> 
>>> # The object has been deleted and so is not on the DB anymore.
>>> WikiPage.query.find({'title': 'MyFirstPage'}).count()
0

In case you need to delete multiple objects and don’t want to query them back to call .delete() on each of them you can use ODMSession.remove() method to delete objects using a query:

>>> WikiPage.query.find().count()
1
>>> WikiPage.query.remove({})
>>> 
>>> WikiPage.query.find().count()
0

Working with SubDocuments

Ming, like MongoDB, allows for documents to be arbitrarily nested. For instance, we might want to keep a metadata property on our WikiPage that kept tag and category information.

To do this, we just declare a WikiPageWithMetadata which will provide a metadata nested document that contains an array of tags and an array of categories. This is made possible by the schema.Object and schema.Array types:

class WikiPageWithMetadata(MappedClass):
    class __mongometa__:
        session = session
        name = 'wiki_page_with_metadata'

    _id = FieldProperty(schema.ObjectId)
    title = FieldProperty(schema.String(required=True))
    text = FieldProperty(schema.String(if_missing=''))

    metadata = FieldProperty(schema.Object({
        'tags': schema.Array(schema.String),
        'categories': schema.Array(schema.String)
    }))

Now, what happens when we create a page and try to update it?

>>> wpm = WikiPageWithMetadata(title='MyPage')
>>> session.flush()
>>> 
>>> # Get back the object to see that Ming creates the correct structure for us
>>> session.clear()
>>> wpm = WikiPageWithMetadata.query.get(title='MyPage')
>>> wpm.metadata
I{'categories': [], 'tags': []}
>>> 
>>> # We can append or edit subdocuments like any other property
>>> wpm.metadata['tags'].append('foo')
>>> wpm.metadata['tags'].append('bar')
>>> session.flush()
>>> 
>>> # Check that ming updated everything on flush
>>> session.clear()
>>> wpm = WikiPageWithMetadata.query.get(title='MyPage')
>>> wpm.metadata
I{'categories': [], 'tags': ['foo', 'bar']}

Ming creates the structure for us automatically. (If we had wanted to specify a different default value for the metadata property, we could have done so using the if_missing parameter, of course.)

Note

In case you want to store complex subdocuments that don’t need validation or for which you don’t want validation for speed reasons you can use the schema.Anything type which can store subdocuments and arrays without validating them.

Relating Classes

The real power of the ODM comes in being able to handle relations between objects. On Ming this is made possible by the ForeignIdProperty and RelationProperty which provide a way to declare references to other documents and retrieve them.

A common use case if for example adding comments on our WikiPage, to do so we need to declare a new WikiComment class:

class WikiComment(MappedClass):
    class __mongometa__:
        session = session
        name = 'wiki_comment'

    _id = FieldProperty(schema.ObjectId)
    page_id = ForeignIdProperty('WikiPage')
    text = FieldProperty(schema.String(if_missing=''))

    page = RelationProperty('WikiPage')

Here, we have defined a ForeignIdProperty page_id to reference the original Wikipage. This tells Ming to create a field in WikiComment that represents a “foreign key” into the WikiPage._id field. This provide a guide to ming on how to resolve the relationship between WikiPage and WikiComment and how to build queries to fetch the related objects.

In order to actually use the relationship, however, we must use a RelationProperty to reference the related class.

In this case, we will use the property page to access the page about which this comment refers. While to access the comments from the WikiPage we will add a comments property to the page itself:

class WikiPage(MappedClass):
    class __mongometa__:
        session = session
        name = 'wiki_page'

    _id = FieldProperty(schema.ObjectId)
    title = FieldProperty(schema.String(required=True))
    text = FieldProperty(schema.String(if_missing=''))

    comments = RelationProperty('WikiComment')

Now actually use these classes, we need to create some comments:

>>> # Get a page for which to add the comments
>>> wp = WikiPage.query.get(title='MyFirstPage')
>>> 
>>> # Create some comments
>>> WikiComment(page_id=wp._id, text='A comment')
<WikiComment _id=ObjectId('6413575b1cb6f0fe17cb481f')
  page_id=ObjectId('6413575b1cb6f0fe17cb481e') text='A
  comment'>
>>> WikiComment(page_id=wp._id, text='Another comment')
<WikiComment _id=ObjectId('6413575b1cb6f0fe17cb4820')
  page_id=ObjectId('6413575b1cb6f0fe17cb481e') text='Another
  comment'>
>>> session.flush()

And voilà, you have related objects. To get them back you can just use the RelationProperty available in the WikiPage as comments:

>>> # Load the original page
>>> wp = WikiPage.query.get(title='MyFirstPage')
>>> 
>>> # View its comments
>>> wp.comments
I[<WikiComment _id=ObjectId('6413575b1cb6f0fe17cb481f')
  page_id=ObjectId('6413575b1cb6f0fe17cb481e') text='A
  comment'>, <WikiComment _id=ObjectId('6413575b1cb6f0fe17cb4820')
  page_id=ObjectId('6413575b1cb6f0fe17cb481e') text='Another
  comment'>]
>>> wp.comments[0].page is wp
True

Relations also support updating by replacing the value of the RelationProperty that drives the relation:

>>> # Load the original page
>>> wp = WikiPage.query.get(title='MyFirstPage')
>>> len(wp.comments)
2
>>> 
>>> wk = WikiComment(text='A comment')
>>> wk.page = wp
>>> session.flush()
>>> 
>>> # Refresh the page to see the relation change even on its side.
>>> wp = session.refresh(wp)
>>> len(wp.comments)
3

While it is not possible to append or remove values from RelationProperty that are a list (like the WikiPage.comments in our example it is still possible to replace its value:

>>> # Load the original page
>>> wp = WikiPage.query.get(title='MyFirstPage')
>>> len(wp.comments)
3
>>> 
>>> wk = WikiComment(text='A comment')
>>> wp.comments = wp.comments + [wk]
>>> session.flush()
>>> 
>>> # Refresh the page to see the relation change even on its side.
>>> wp = session.refresh(wp)
>>> len(wp.comments)
4
>>> # Refresh the comment to see the relation change even on its side.
>>> wk = session.refresh(wk)
>>> wk.page is wp
True

Note

You should treat ming relations that contain more than one value as tuples, they are stored as an immutable list.

List based Relationships (Many-to-Many)

Ming supports many-to-many relationships by using MongoDB arrays. Instead of storing the foreign id as an ObjectId it can be storead as an array of ObjectId.

This can be achieved by using the uselist=True option of ForeignIdProperty:

class Parent(MappedClass):
    class __mongometa__:
        name='parent'
        session = session

    _id = FieldProperty(schema.ObjectId)
    name = FieldProperty(schema.String(required=True))
    _children = ForeignIdProperty('Child', uselist=True)

    children = RelationProperty('Child')

class Child(MappedClass):
    class __mongometa__:
        name='child'
        session = session

    _id = FieldProperty(schema.ObjectId)
    name = FieldProperty(schema.String(required=True))

    parents = RelationProperty('Parent')

Then you can create and relate objects as you would with any other kind of relationship:

>>> # Create a bunch of Parents and Children
>>> p1, p2 = Parent(name='p1'), Parent(name='p2')
>>> c1, c2 = Child(name='c1'), Child(name='c2')
>>> 
>>> # Relate them through their relationship
>>> p1.children = [c1, c2]
>>> c2.parents = [p1, p2]
>>> session.flush()
>>> 
>>> # Fetch them back to see if the relations are correct
>>> session.clear()
>>> p1, p2 = Parent.query.find({}).sort('name').all()
>>> c1, c2 = Child.query.find({}).sort('name').all()
>>> 
>>> len(c1.parents)  # c1 was only assigned to p1
1
>>> len(c2.parents)  # c2 was assigned both to p1 and p2
2
>>> len(p1.children) == 2 and c1 in p1.children
True
>>> len(p2.children) == 1 and p2.children[0] is c2
True

Forcing a ForeignIdProperty

By default the RelationProperty will automatically detect the relationship side and foreign id property. In case the automatic detection fails it is possible to manually specify it through the via="propertyname" option of the RelationProperty:

class WebSite(MappedClass):
    class __mongometa__:
        name='web_site'
        session = self.session

    _id = FieldProperty(schema.ObjectId)

    _index = ForeignIdProperty('WebPage')
    index = RelationProperty('WebPage', via='_index')

    pages = RelationProperty('WebPage', via='_website')

class WebPage(MappedClass):
    class __mongometa__:
        name='web_page'
        session = self.session

    _id = FieldProperty(schema.ObjectId)

    _website = ForeignIdProperty('WebSite')
    website = RelationProperty('WebSite', via='_website')

Forcing a Relationship side

While it is possible to force a specific ForeignIdProperty there are cases when it is also necessary to force the relationship side. This might be the case if both sides of the relationship own a property with the specified name, a common case is a circular relationship where both sides are the same class.

Specifying the side is possible by passing a tuple to the via property with the second value being True or False depending on the fact that the specified ForeignIdProperty should be considered owned by this side of the relationship or not:

class WebPage(MappedClass):
    class __mongometa__:
        name='web_page'
        session = self.session

    _id = FieldProperty(int)

    children = RelationProperty('WebPage')
    _children = ForeignIdProperty('WebPage', uselist=True)
    parents = RelationProperty('WebPage', via=('_children', False))

Dropping Down Below the ODM

There might be cases when you want to directly access the pymongo layer or get outside of the ODM for speed reasons or to get access to additional features.

This can be achieved by dropping down to the Ming Foundation Layer through the ODM Mapper. The Foundation Layer is a lower level API that provides validation and schema enforcement over plain dictionaries. The Mapper has a collection attribute that points to the Foundation Layer collection. The Foundation Layer adds additional features over plain dictionaries through the m property:

>>> from ming.odm import mapper
>>> mongocol = mapper(WikiPage).collection.m.collection
>>> mongocol
mim.Collection(mim.Database(odm_tutorial), wiki_page)
>>> 
>>> mongocol.find_one({'title': 'MyFirstPage'})
{'_id': ObjectId('6413575b1cb6f0fe17cb481e'), 'text': 'This is my first page!!!', 'title': 'MyFirstPage'}

You can also operate at the Foundation Layer through the Mapper and ODMSession by accessing the low level Session implementation and mapped Collection:

>>> from ming.odm import mapper
>>> wikipage_mapper = mapper(WikiPage)
>>> 
>>> # Mapper.collection is the foundation layer collection
>>> founding_WikiPage = wikipage_mapper.collection
>>> 
>>> # Retrieve the foundation layer session
>>> founding_Session = session.impl
>>> 
>>> # The foundation layer still returns dictionaries, but validation is performed.
>>> founding_Session.find(founding_WikiPage, {'title': 'MyFirstPage'}).all()
[{'_id': ObjectId('6413575b1cb6f0fe17cb481e'), 'text': 'This is my first page!!!', 'title': 'MyFirstPage'}]

Ensuring Indexing

The ODM layer permits to ensure indexes over the collections by using the __mongometa__ attribute. You can enforce both unique indexing and non-unique indexing.

To apply the indexes sepcified in your models you can then iterate over all the mappers and call Mapper.ensure_all_indexes() to guarantee that indexes are created for every registered mapper:

ming.odm.Mapper.ensure_all_indexes()

This needs to be performed each time you change the indexes or the database. It is common practice to ensure all the indexes at application startup.

Indexing a Field

Indexing a field is possible by just setting index=True parameter when creating a FieldProperty:

class Permission(MappedClass):
    class __mongometa__:
        session = session
        name = 'permissions'

    _id = FieldProperty(s.ObjectId)
    permission_name = FieldProperty(s.String, index=True)
    description = FieldProperty(s.String)
    groups = FieldProperty(s.Array(str))

More flexible indexes definition is also available through the indexes property of __mongometa__:

class Permission(MappedClass):
    class __mongometa__:
        session = session
        name = 'permissions'
        indexes = [('permission_name', )]

    _id = FieldProperty(s.ObjectId)
    permission_name = FieldProperty(s.String)
    description = FieldProperty(s.String)
    groups = FieldProperty(s.Array(str))

Indexes are represented by tuples which can have one or more entries (to represent compound indexes):

indexes = [('permission_name', 'groups'),]

Also the tuples can contain a (field, direction) declaration to indicate where the index is sorted:

indexes = [
    (('permission_name', ming.ASCENDING), ('groups', ming.DESCENDING))
]

Multiple indexes can be declared for a collection:

indexes = [
    (('permission_name', ming.ASCENDING), ('groups', ming.DESCENDING)),
    ('groups', )
]

Unique Indexes

You can use unique_indexes to ensure that it won’t be possible to duplicate an object multiple times with the same name:

class Permission(MappedClass):
    class __mongometa__:
        session = session
        name = 'permissions'
        unique_indexes = [('permission_name',)]

    _id = FieldProperty(s.ObjectId)
    permission_name = FieldProperty(s.String)
    description = FieldProperty(s.String)
    groups = FieldProperty(s.Array(str))

Custom Indexes

If you want more control over your indexes, you can use custom_indexes directly within __mongometa__, this will allow you to explicitly set unique and/or sparse index flags that same way you could if you were directly calling ensureIndex in the MongoDB shell. For example, if you had a field like email that you wanted to be unique, but also allow for it to be Missing

class User(MappedClass):
    class __mongometa__:
        session = session
        name = 'users'
        custom_indexes = [
            dict(fields=('email',), unique=True, sparse=True)
        ]

    _id = FieldProperty(s.ObjectId)
    email = FieldProperty(s.String, if_missing=s.Missing)

Indexes definitions in custom_indexes can actually contain any key which is a valid argument for Index initialization function as they are used to actually create Index instances.

Now when accessing instances of User, if email is Missing and you attempt to use the User.email attribute Ming, will throw an AttributeError as it ensures that only properties that are not Missing are mapped as attributes to the class.

This brings us to the ming.odm.property.FieldPropertyWithMissingNone property type. This allows you to mimic the behavior that you commonly find in a SQL solution. An indexed and unique field that is also allowed to be NULL or in this case Missing. A classic example would be a product database where you want to enter in products but don’t have SKU numbers for them yet. Now your product listing can still call product.sku without throwing an AttributeError.

class Product(MappedClass):
    class __mongometa__:
        session = session
        name = 'products'
        custom_indexes = [
            dict(fields=('sku',), unique=True, sparse=True)
        ]

    _id = FieldProperty(s.ObjectId)
    sku = FieldPropertyWithMissingNone(str, if_missing=s.Missing)

Model Evolution and Migrations

One of the most irritating parts of maintaining an application for a while is the need to do data migrations from one version of the schema to another. While Ming can’t completely remove the pain of migrations, it does seek to make migrations as simple as possible.

Performing Migrations

First of all let’s populate our database with some stray data that needs to be migrated:

>>> import random
>>> TAGS = ['foo', 'bar', 'snafu', 'mongodb']
>>> 
>>> # Insert the documents through PyMongo so that Ming is not involved
>>> session.db.wiki_page.insert([
...     dict(title='Page %s' % idx, text='Text of Page %s' %idx, tags=random.sample(TAGS, 2)) for idx in range(10)
... ])
[ObjectId('6413575b1cb6f0fe17cb480d'), ObjectId('6413575b1cb6f0fe17cb480e'), ObjectId('6413575b1cb6f0fe17cb480f'), ObjectId('6413575b1cb6f0fe17cb4810'), ObjectId('6413575b1cb6f0fe17cb4811'), ObjectId('6413575b1cb6f0fe17cb4812'), ObjectId('6413575b1cb6f0fe17cb4813'), ObjectId('6413575b1cb6f0fe17cb4814'), ObjectId('6413575b1cb6f0fe17cb4815'), ObjectId('6413575b1cb6f0fe17cb4816')]
>>> 
>>> session.db.wiki_page.find_one()
{'_id': ObjectId('6413575b1cb6f0fe17cb480d'), 'title': 'Page 0', 'text': 'Text of Page 0', 'tags': ['mongodb', 'foo']}

Suppose we decided that we want to gather metadata of the pages in a metadata property, which will contain the categories and tags of the page. We might write our new schema as follows:

class WikiPage(MappedClass):
    class __mongometa__:
        session = session
        name = 'wiki_page'

    _id = FieldProperty(schema.ObjectId)
    title = FieldProperty(schema.String(required=True))
    text = FieldProperty(schema.String(if_missing=''))

    metadata = FieldProperty(schema.Object({
        'tags': schema.Array(schema.String),
        'categories': schema.Array(schema.String)
    }))

But now if we try to .find() things in our database, our metadata has gone missing:

>>> WikiPage.query.find().first()
<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb480d')
  title='Page 0' text='Text of Page 0' metadata=I{'tags':
  [], 'categories': []}>

What we need now is a migration. Luckily, Ming makes migrations manageable.

First of all we need to declare the previous schema so that Ming knows how to validate the old values (previous versions schemas are declared using the Ming Foundation Layer as they are not tracked by the UnitOfWork or IdentityMap):

from ming import collection, Field

OldWikiPageCollection = collection('wiki_page', session,
    Field('_id', schema.ObjectId),
    Field('title', schema.String),
    Field('text', schema.String),
    Field('tags', schema.Array(schema.String))
)

Whenever Ming fetches a document from the database it will validate it against our model schema.

If the validation fails it will check the document against the previous version of the schema (provided as __mongometa__.version_of) and if validation passes the __mongometa__.migrate function is called to upgrade the data.

So, to be able to upgrade our data, all we need to do is include the previous schema, and a migration function in our __mongometa__:

class WikiPage(MappedClass):
    class __mongometa__:
        session = session
        name = 'wiki_page'
        version_of = OldWikiPageCollection

        @staticmethod
        def migrate(data):
            result = dict(data, metadata={'tags': data['tags']}, _version=1)
            del result['tags']
            return result

    _id = FieldProperty(schema.ObjectId)
    title = FieldProperty(schema.String(required=True))
    text = FieldProperty(schema.String(if_missing=''))

    _version = FieldProperty(1, required=True)

    metadata = FieldProperty(schema.Object({
        'tags': schema.Array(schema.String),
        'categories': schema.Array(schema.String)
    }))

Then to force the migration we also added a _version property which passes validation only when its value is 1 (Using schema.Value). As old models do not provide a _version field they won’t pass validation and so they will trigger the migrate process:

>>> WikiPage.query.find().limit(3).all()
[<WikiPage _id=ObjectId('6413575b1cb6f0fe17cb480d')
  title='Page 0' text='Text of Page 0' _version=1
  metadata=I{'categories': [], 'tags': ['mongodb', 'foo']}>, <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb480e')
  title='Page 1' text='Text of Page 1' _version=1
  metadata=I{'categories': [], 'tags': ['foo', 'snafu']}>, <WikiPage _id=ObjectId('6413575b1cb6f0fe17cb480f')
  title='Page 2' text='Text of Page 2' _version=1
  metadata=I{'categories': [], 'tags': ['bar', 'foo']}>]

And that’s it.

Lazy Migrations

Migrations are performed lazily as the objects are loaded from the database, so you only pay the cost of migration the data you access. Also the migrated data is not saved back on the database unless the object is modified. This can be easily seen by querying documents directly through pymongo as on mongodb they still have tags outside of metadata:

>>> next(session.db.wiki_page.find())
{'_id': ObjectId('6413575b1cb6f0fe17cb480d'), 'title': 'Page 0', 'text': 'Text of Page 0', 'tags': ['mongodb', 'foo']}

Eager Migrations

If, unlike for lazy migrations, you wish to migrate all the objects in a collection, and save them back you can use the migrate function available on the foundation layer manager:

>>> next(session.db.wiki_page.find()).get('tags')
['mongodb', 'foo']
>>> 
>>> from ming.odm import mapper
>>> mapper(WikiPage).collection.m.migrate()
>>> 
>>> next(session.db.wiki_page.find()).get('metadata')
{'categories': [], 'tags': ['mongodb', 'foo']}

That will automatically migrate all the documents in the collection one by one.

Chained Migrations

If you evolved your schema multiple times you can chain migrations by adding a version_of to all the previous versions of the data:

class MyModel(MappedClass):
    class __mongometa__:
        session = session
        name = 'mymodel'
        version_of = collection('mymodel', session,
            Field('_id', schema.ObjectId),
            Field('name', schema.String),
            Field('_version', schema.Value(1, required=True)),
            version_of=collection('mymodel', session,
                Field('_id', schema.ObjectId),
                Field('name', schema.String),
            ),
            migrate=lambda data: dict(_id=data['_id'], name=data['name'].upper(), _version=1)
        )

        @staticmethod
        def migrate(data):
            return dict(_id=data['_id'], name=data['name'][::-1], _version=2)

    _id = FieldProperty(schema.ObjectId)
    name = FieldProperty(schema.String(required=True))

    _version = FieldProperty(2, required=True)

Then just apply all the migrations as you normally would:

>>> session.db.mymodel.insert(dict(name='desrever'))
[ObjectId('6413575b1cb6f0fe17cb4817')]
>>> session.db.mymodel.find_one()
{'_id': ObjectId('6413575b1cb6f0fe17cb4817'), 'name': 'desrever'}
>>> 
>>> # Apply migration to version 1 and then to version 2
>>> mapper(MyModel).collection.m.migrate()
>>> 
>>> session.db.mymodel.find_one()
{'_id': ObjectId('6413575b1cb6f0fe17cb4817'), '_version': 2, 'name': 'REVERSED'}

The resulting documented changed name from "desrever" to "REVERSED" that is because _version=1 forced the name to be uppercase and then _version=2 reversed it.

Note

When migrating make sure you always bring forward the _id value in the old data, or you will end up with duplicated data for each migration step as a new id would be generated for newly migrated documents.

ODM Event Interfaces

This section describes the various categories of events which can be intercepted within the Ming ODM. Events can be trapped by registering MapperExtension and SessionExtension instances which implement the event handlers for the events you want to trap.

Mapper Events

Mapper events are used to track To use MapperExtension, make your own subclass of it and just send it off to a mapper:

from ming.odm.mapper import MapperExtension
class MyExtension(MapperExtension):
    def after_insert(self, obj, st, sess):
        print "instance %s after insert !" % obj

class MyMappedClass(MappedClass):
    class __mongometa__:
        session = session
        name = 'my_mapped_class'
        extensions = [ MyExtension ]

Multiple extensions will be chained together and processed in order;

extensions = [ext1, ext2, ext3]

class ming.odm.MapperExtension(mapper)

Base class that should be inherited to handle Mapper events.

after_delete(instance, state, sess)

Receive an object instance and its current state after that instance is deleted.

after_insert(instance, state, sess)

Receive an object instance and its current state after that instance is inserted into its collection.

after_remove(sess, *args, **kwargs)

After a remove query is performed for this class

after_update(instance, state, sess)

Receive an object instance and its current state after that instance is updated.

before_delete(instance, state, sess)

Receive an object instance and its current state before that instance is deleted.

before_insert(instance, state, sess)

Receive an object instance and its current state before that instance is inserted into its collection.

before_remove(sess, *args, **kwargs)

Before a remove query is performed for this class

before_update(instance, state, sess)

Receive an object instance and its current state before that instance is updated.

Session Events

The SessionExtension applies plugin points for Session objects and ODMCursor objects:

from ming.odm.base import state
from ming.odm.odmsession import SessionExtension

class MySessionExtension(SessionExtension):
    def __init__(self, session):
        SessionExtension.__init__(self, session)
        self.objects_added = []
        self.objects_modified = []
        self.objects_deleted = []

    def before_flush(self, obj=None):
        if obj is None:
            self.objects_added = list(self.session.uow.new)
            self.objects_modified = list(self.session.uow.dirty)
            self.objects_deleted = list(self.session.uow.deleted)
        # do something

ODMSession = ThreadLocalODMSession(session,
                                   extensions=[ProjectSessionExtension])

The same SessionExtension instance can be used with any number of sessions. It is possible to register extensions on an already created ODMSession using the register_extension(extension) method of the session itself. Even calling register_extension it is possible to register the extensions only before using the session for the first time.

class ming.odm.SessionExtension(session)

Base class that should be inherited to handle Session events.

after_cursor_next(cursor)

Cursor has advanced to next result

after_delete(obj, st)

After an object gets deleted in this session

after_flush(obj=None)

After the session is flushed for obj

If obj is None it means all the objects in the UnitOfWork which can be retrieved by iterating over ODMSession.uow

after_insert(obj, st)

After an object gets inserted in this session

after_remove(cls, *args, **kwargs)

After a remove query is performed session

after_update(obj, st)

After an object gets updated in this session

before_cursor_next(cursor)

Cursor is going to advance to next result

before_delete(obj, st)

Before an object gets deleted in this session

before_flush(obj=None)

Before the session is flushed for obj

If obj is None it means all the objects in the UnitOfWork which can be retrieved by iterating over ODMSession.uow

before_insert(obj, st)

Before an object gets inserted in this session

before_remove(cls, *args, **kwargs)

Before a remove query is performed session

before_update(obj, st)

Before an object gets updated in this session

cursor_created(cursor, action, *args, **kw)

New cursor with the results of a query got created

Polymorphic Entities

Introduction

Polymorphic entities are entities that inherit from the same base model, are stored in the same collection but can have different properties and behaviours depending on a guard value.

They are usually available to provide inheritance and specialised behaviours in models as you would for Python subclasses.

Declaring Polymorphic Entities

First we need to declare our base model which will provide the common properties and the property on which the model identity is recognized:

class Transport(MappedClass):
    class __mongometa__:
        session = session
        name = 'transport'
        polymorphic_on = '_type'
        polymorphic_identity = 'base'

    _id = FieldProperty(schema.ObjectId)
    origin = FieldProperty(schema.String(required=True))
    destination = FieldProperty(schema.String(if_missing=''))
    _type = FieldProperty(schema.String(if_missing='base'))

    def move(self):
        return f'moving from {self.origin} to {self.destination}'

The _type property is used to store the identity of the model, we usually won’t be writing this property, it will be automatically filled using if_missing with a different value for each subclass.

__mongometa__.polymorphic_on attribute is used to tell Ming on which property the polymorphism is happening and __mongometa__.polymorphic_identity will be provided by each class to tell which value of _type is bound to that class:

class Bus(Transport):
    class __mongometa__:
        polymorphic_identity = 'bus'

    _type=FieldProperty(str, if_missing='bus')
    passengers_count = FieldProperty(schema.Int(if_missing=0))

    def move(self):
        return f'driving from {self.origin} to {self.destination}'

class AirBus(Bus):
    class __mongometa__:
        polymorphic_identity = 'airbus'

    _type=FieldProperty(str, if_missing='airbus')
    wings_count = FieldProperty(schema.Int(if_missing=2))

    def move(self):
        return f'flying from {self.origin} to {self.destination}'

Now we can create Bus or AirBus instances freely and use the additional properties provided by each one of them:

>>> # Create 2 Bus
>>> Bus(origin='Rome', destination='London', passengers_count=20)
<Bus _type='bus' passengers_count=20
  _id=ObjectId('6413575b1cb6f0fe17cb4818') origin='Rome'
  destination='London'>
>>> Bus(origin='Turin', destination='London', passengers_count=20)
<Bus _type='bus' passengers_count=20
  _id=ObjectId('6413575b1cb6f0fe17cb4819') origin='Turin'
  destination='London'>
>>> # And an AirBus
>>> AirBus(origin='Turin', destination='London', passengers_count=60, wings_count=3)
<AirBus _type='airbus' wings_count=3 passengers_count=60
  _id=ObjectId('6413575b1cb6f0fe17cb481a') origin='Turin'
  destination='London'>
>>> session.flush()

When querying them back we can see that they all ended up in the Transport collection and that as expected only two Bus instances got created (the third is an AirBus):

>>> session.clear()
>>> Transport.query.find().count()
3
>>> Transport.query.find({'_type': 'bus'}).count()
2

Querying Polymorphic Entities

When querying back polymorphic entities you should always query them back from their base type (in this case Transport) as the type promotion will be automatically performed by ming:

>>> Transport.query.find().all()
[<Bus _type='bus' passengers_count=20
  _id=ObjectId('6413575b1cb6f0fe17cb4818') origin='Rome'
  destination='London'>, <Bus _type='bus' passengers_count=20
  _id=ObjectId('6413575b1cb6f0fe17cb4819') origin='Turin'
  destination='London'>, <AirBus _type='airbus' wings_count=3 passengers_count=60
  _id=ObjectId('6413575b1cb6f0fe17cb481a') origin='Turin'
  destination='London'>]

As types are properly promoted it is also possible to rely on behaviours specific of each single subclass:

>>> transports = Transport.query.find().all()
>>> transports[0].move()
'driving from Rome to London'
>>> transports[1].move()
'driving from Turin to London'
>>> transports[2].move()
'flying from Turin to London'

Custom Schemas and Properties

Ming provides builtin properties for most common use cases: RelationProperty, ForeignIdProperty, FieldProperty, FieldPropertyWithMissingNone those allow you to define relations, references to other models, store common values and get back properties that might not be available on the collection.

Also when validating data the ming.schema module provides schemas to validate most common use cases and data types.

While those are usually enough, there might be cases where you want to define your own properties or schemas to force better contraints or add conversions.

Custom Schema

Schemas are only used to enforce constraints, they usually do not convert values and are applied both when data is loaded from or saved to the database.

Validating with Schemas

Schemas can be easily implemented by subclassing FancySchemaItem which already provides if_missing and required support. Then the actual validation will be performed by a custom ._validate method:

class EmailSchema(schema.FancySchemaItem):
    regex = re.compile(r'[\w\.\+\-]+\@[\w]+\.[a-z]{2,3}$')

    def _validate(self, value, **kw):
        if not self.regex.match(value):
            raise schema.Invalid('Not a valid email address', value)
        return value

Then objects can use that schema like any other:

class Contact(MappedClass):
    class __mongometa__:
        session = session
        name = 'contact'

    _id = FieldProperty(schema.ObjectId)
    name = FieldProperty(schema.String(required=True))
    email = FieldProperty(EmailSchema)

And can be created and queried as usual:

>>> Contact(name='Contact 1', email='contact1@server.com')
<Contact _id=ObjectId('6413575b1cb6f0fe17cb4809')
  name='Contact 1' email='contact1@server.com'>
>>> session.flush()
>>> session.clear()
>>> 
>>> c = Contact.query.find().first()
>>> c.email
'contact1@server.com'

Trying to create a Contact with an invalid email address will fail as it won’t pass our schema validation:

>>> try:
...     Contact(name='Contact 1', email='this-is-invalid')
... except schema.Invalid as e:
...     error = e
... 
>>> error
Invalid('Not a valid email address')

Schema validation is not only enforced on newly created items or when setting properties, but also on data loaded from the database. This is because as MongoDB is schema-less there is no guarantee that the data we are loading back is properly formatted:

>>> session.db.contact.insert(dict(name='Invalid Contact',
...                                email='this-is-invalid'))
[ObjectId('6413575b1cb6f0fe17cb480a')]
>>> 
>>> try:
...     c1 = Contact.query.find().first()
... except schema.Invalid as e:
...     error = e
... 
>>> error
Invalid('email:Not a valid email address')

Schemas can’t convert

As schemas are validated both when saving and loading data a good use case for a custom schema might be checking that a field is storing a properly formatted email address but it cannot be used for an hashed password.

Let’s see what happens when we try to perform some kind of conversion inside a schema. For example we might try to define a PasswordSchema:

class PasswordSchema(schema.FancySchemaItem):
    def _validate(self, value, **kwargs):
        return hashlib.md5(value).hexdigest()

which is used by our UserWithSchema class:

class UserWithSchema(MappedClass):
    class __mongometa__:
        session = session
        name = 'user_with_schema'

    _id = FieldProperty(schema.ObjectId)
    name = FieldProperty(schema.String(required=True))
    password = FieldProperty(PasswordSchema)

Then we can create a new user and query it back:

>>> user = UserWithSchema(name='User 1',
...                       password='12345678')
Traceback (most recent call last):
  File "<console>", line 2, in <module>
  File "/home/docs/checkouts/readthedocs.org/user_builds/ming/envs/latest/lib/python3.7/site-packages/Ming-0.13.0-py3.7.egg/ming/odm/mapper.py", line 412, in __init__
    self.func(self_, *args, **kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ming/envs/latest/lib/python3.7/site-packages/Ming-0.13.0-py3.7.egg/ming/odm/mapper.py", line 446, in _basic_init
    setattr(self_, k, v)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ming/envs/latest/lib/python3.7/site-packages/Ming-0.13.0-py3.7.egg/ming/odm/property.py", line 93, in __set__
    value = self.field.schema.validate(value)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ming/envs/latest/lib/python3.7/site-packages/Ming-0.13.0-py3.7.egg/ming/schema.py", line 268, in _validate_optional
    return self._validate(value, **kw)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ming/checkouts/latest/docs/src/ming_odm_schemas.py", line 37, in _validate
    return hashlib.md5(value).hexdigest()
TypeError: Unicode-objects must be encoded before hashing
>>> session.flush()
>>> session.clear()
>>> 
>>> user = UserWithSchema.query.find().first()
>>> user.password
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'password'

At first sight it might seem that everything worked as expected. Our user got created and the password is actually an md5.

But is it the right md5?

>>> user = UserWithSchema.query.find().first()
>>> hashlib.md5('12345678').hexdigest() == user.password
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeError: Unicode-objects must be encoded before hashing

It looks like it isn’t. Actually when we query the user back we get something which is even different from the value stored on the database:

>>> user = UserWithSchema.query.find().first()
>>> user.password
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'password'
>>> 
>>> user = session.db.user_with_schema.find_one()
>>> user['password']
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable

And what we have on the db is not even the md5 of our password:

>>> user = session.db.user_with_schema.find_one()
>>> user['password']
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable
>>> 
>>> hashlib.md5('12345678').hexdigest()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeError: Unicode-objects must be encoded before hashing

That’s because our value is actually the md5 recursively applied multiple times whenever the validation was performed:

>>> user = UserWithSchema.query.find().first()
>>> 
>>> pwdmd5 = hashlib.md5('12345678').hexdigest()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeError: Unicode-objects must be encoded before hashing
>>> for i in range(10):
...     if pwdmd5 == user.password:
...         break
...     pwdmd5 = hashlib.md5(pwdmd5).hexdigest()
... 
Traceback (most recent call last):
  File "<console>", line 2, in <module>
NameError: name 'pwdmd5' is not defined
>>> pwdmd5 == user.password
Traceback (most recent call last):
  File "<console>", line 1, in <module>
NameError: name 'pwdmd5' is not defined
>>> i
0

So what we learnt is that schemas should never be used to convert values as they can be applied recursively any number of times whenever the document is saved or loaded back!

So what can we use to convert values? Custom Properties

Custom Properties

Custom Properties are specific to the ODM layer and are not available on the Ming Foundation Layer which implements only schema validation.

The benefit of custom properties over schemas is that you actually know whenever the valid is read or saved and so they can be properly used for conversion of values to and from python.

Converting with Properties

Ming Properties actually implement the Python Descriptor Protocol which is based on __get__, __set__?and __delete__ methods to retrieve, save and remove values from an object.

So implementing a custom property is a matter of subclassing FieldProperty and providing our custom behaviour:

class PasswordProperty(FieldProperty):
    def __init__(self):
        # Password is always a required string.
        super().__init__(schema.String(required=True))

    def __get__(self, instance, cls=None):
        if instance is None: return self

        class Password(str):
            def __new__(cls, content):
                self = str.__new__(cls, '******')
                self.raw_value = content
                return self

        # As we don't want to leak passwords we return an asterisked string
        # but the real value of the password will always be available as .raw_value
        # so we can check passwords when logging in.
        return Password(super().__get__(instance, cls))

    def __set__(self, instance, value):
        pwd = hashlib.md5(value).hexdigest()
        super().__set__(instance, pwd)

Then we can use it like any other property in our model:

class User(MappedClass):
    class __mongometa__:
        session = session
        name = 'user'

    _id = FieldProperty(schema.ObjectId)
    name = FieldProperty(schema.String(required=True))
    password = PasswordProperty()

This is already enough to be able to store properly hashed passwords:

>>> user = User(name='User 1',
...             password='12345678')
Traceback (most recent call last):
  File "<console>", line 2, in <module>
  File "/home/docs/checkouts/readthedocs.org/user_builds/ming/envs/latest/lib/python3.7/site-packages/Ming-0.13.0-py3.7.egg/ming/odm/mapper.py", line 412, in __init__
    self.func(self_, *args, **kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ming/envs/latest/lib/python3.7/site-packages/Ming-0.13.0-py3.7.egg/ming/odm/mapper.py", line 446, in _basic_init
    setattr(self_, k, v)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ming/checkouts/latest/docs/src/ming_odm_properties.py", line 37, in __set__
    pwd = hashlib.md5(value).hexdigest()
TypeError: Unicode-objects must be encoded before hashing
>>> session.flush()
>>> 
>>> user = session.db.user.find_one()
>>> user['password']
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable
>>> user['password'] == hashlib.md5('12345678').hexdigest()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable

And as we provided some kind of password leakage prevention by always returning an asterisked string for the password let’s check if it works as expected:

>>> user = User.query.find().first()
>>> user.password
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'password'
>>> user.password.raw_value
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'password'

As we can see the password is properly returned as a Password instance which is a string with asterisk that also provides the real value as .raw_value.

Ming Foundation Layer

MongoDB and Ming

MongoDB is a high-performance schemaless database that allows you to store and retrieve JSON-like documents. MongoDB stores these documents in collections, which are analogous to SQL tables. Because MongoDB is schemaless, there are no guarantees given to the database client of the format of the data that may be returned from a query; you can put any kind of document into a collection that you want.

While this dynamic behavior is handy in a rapid development environment where you might delete and re-create the database many times a day, it starts to be a problem when you need to make guarantees of the type of data in a collection (because you code depends on it). The goal of Ming is to allow you to specify the schema for your data in Python code and then develop in confidence, knowing the format of data you get from a query.

As Ming is heavily inspired by SQLAlchemy its Object Document Mapper layer is mapped over a foundation layer (through the Mapper class) that provides the basic validation and connection management features over dictionaries.

Connecting to the Database

Ming manages your connection to the MongoDB database using an object known as a DataStore. The DataStore is actually just a thin wrapper around a pymongo Database object which is used by ming.Session to perform the actual queries:

from ming import Session
session = Session(create_datastore('mongodb://localhost:27017/tutorial'))

Note

Note that Session is thread-safe by itself as it stores no data, differently from the base ODMSession class which requires to be used through ThreadLocalODMSession to be thread safe.

Collection objects

Declarative versus Imperative

There are two styles to define models in Ming at foundation layer: declarative and imperative. Both styles are available at the document level (which this tutorial covers). Which you end up using is mostly a matter of personal style. The declarative style actually predated the imperative style, and the main author of Ming uses both styles interchangeably in application programming based on which seems more convenient for the task at hand.

Now that that boilerplate is out of the way, we can actually start writing our models. We will start with a model representing a WikiPage. We can do that in “imperative” mode as follows:

from ming import collection, Field, schema

WikiPage = collection('wiki_page', session,
    Field('_id', schema.ObjectId),
    Field('title', schema.String),
    Field('text', schema.String)
)

Here, we define a WikiPage Python class with three fields, _id, title, and text. We also bind the WikiPage to the session we defined earlier and to the wiki_page name.

If you prefer a “declarative” style, you can also declare a collection by subclassing the Document class:

from ming import Field, schema
from ming.declarative import Document

class WikiPage(Document):

    class __mongometa__:
        session = session
        name = 'wiki_page'

    _id = Field(schema.ObjectId)
    title = Field(schema.String)
    text = Field(schema.String)

Here, rather than use the collection() function, we are defining the class directly, grouping some of the metadata used by ming into a __mongometa__ class in order to reduce namespace conflicts. Note that we don’t have to provide the name of our various Field instances as strings here since they already have names implied by their names as class attributes. If we want to map a document field to a different class attribute, we can do so using the following syntax:

_renamed_field = Field('renamed_field', schema.String)

This is sometimes useful for “privatizing” document members that we wish to wrap in @property decorators or other access controls.

We can add our own methods to the WikiPage class, too. However, the make() method is reserved for object construction and validation. See the Bad Data section.

Type Annotations

Some type annotations are in Ming, but you need to add a hint to each class to help. You must be using the “declarative” approach that inherits from Document. The primary goal so far is to improve IDE experience. They may or may not work with mypy. Add some imports and the m: line to your models like this:

import typing

if typing.TYPE_CHECKING:
    from ming.metadata import Manager

...

class WikiPage(Document):

    class __mongometa__:
        session = session
        name = 'wiki_page'

    m: 'Manager[WikiPage]'

    ...

Using Ming Objects to Represent Mongo Records

Now that we’ve defined a basic schema, let’s start playing around with Ming in the interactive interpreter. First, make sure you’ve saved the code below in a module “tutorial.py”:

from ming import Session, create_datastore
from ming import Document, Field, schema

bind = create_datastore('tutorial')
session = Session(bind)

class WikiPage(Document):

    class __mongometa__:
        session = session
        name = 'wiki_page'

    _id = Field(schema.ObjectId)
    title = Field(str)
    text = Field(str)

Now let’s fire up the interpreter and start working. The first thing we’ll do is create a WikiPage:

>>> page = WikiPage(dict(title='MyPage', text=''))
>>> page
{'text': '', 'title': 'MyPage'}
>>> page.title
'MyPage'
>>> page['title']
'MyPage'

As you can see, Ming documents can be accessed either using dictionary-style lookups (page[‘title’]) or attribute-style lookups (page.title). In fact, all Ming documents are dict subclasses, so all the standard methods on Python ict objects are available.

In order to actually interact with the database, Ming provides a standard attribute .m, short for Manager, on each mapped class.

In order to save the document we just created to the database, for instance, we would simply type:

>>> page.m.save()
>>> page
{'text': '', '_id': ObjectId('4b1d638ceb033028a0000000'), 'title': 'MyPage'}

When the page was saved to the database, the database assigned a unique _id attribute. (If we had wished to specify our own _id, we could have also done that.) Now, let’s query the database and make sure that the document actually got saved:

>>> WikiPage.m.find().first()
{'text': u'', '_id': ObjectId('4b1d638ceb033028a0000000'), 'title': u'MyPage'}

And there it is! Now, let’s add some text to the page:

>>> page.text = 'This is some text on my page'
>>> page.m.save()
>>> WikiPage.m.find().first()
{'text': u'This is some text on my page', '_id': ObjectId('4b1d638ceb033028a0000000'), 'title': u'MyPage'}

Looks like it worked. One thing we glossed over was the use of the .m.find() method. This is the main method we’ll use to query the database, and is covered in the next section.

Querying the Database

Ming provides an .m attribute that exposes the same methods available on Session just bound to the Collection or instance of your Documents.

The .m.find() method works just like the .find() method on collection objects in pymongo and is used for performing queries.

The result of a query is a Python iterator that wraps a pymongo cursor, converting each result to a ming.Document before yielding it.

Like SQLAlchemy, we provide several convenience methods on query results through Cursor:

one()

Retrieve a single result from a query. Raises an exception if the query contains either zero or more than one result.

first()

Retrieve the first result from a query. If there are no results, return None.

all()

Retrieve all results from a query, storing them in a Python list.

count()

Returns the number of results in a query

limit(limit)

Restricts the cursor to only return limit results

skip(skip)

Skips ahead skip results in the cursor (similar to a SQL OFFSET clause)

sort(*args, **kwargs)

Sorts the underlying pymongo cursor using the same semantics as the pymongo.Cursor.sort() method

Ming also provides a convenience method .m.get(**kwargs) which is equivalent to .m.find(kwargs).first() for simple queries that are expected to return one result. Some examples:

>>> WikiPage.m.find({'title': 'MyPage'}).first()
{'text': u'', '_id': ObjectId('4b1d638ceb033028a0000000'), 'title': u'MyPage'}
>>> WikiPage.m.find().count()
1
>>> WikiPage.m.get(title='MyPage')
{'text': u'', '_id': ObjectId('4b1d638ceb033028a0000000'), 'title': u'MyPage'}

Other Sessions

If we have a special case where we want to use a different database session for a model, other than the one specified in __mongometa__, we can do:

foobar = Session.by_name('foobar')
foobar.save(my_model_instance)

or:

foobar = Session.by_name('foobar')
my_model_instance.m(foobar).save()

This could be useful if you have a database session that is connected to a master server, and another one that is used for the slave (readonly).

Bad Data

Schema Validation

Ming documents are validated at certain points in their life cycle. (Validation is where the schema is enforced on the document.) Generally, schema validation occurs when saving the document to the database or when loading it from the database. Additionally, validation is performed when the document is created using the .make() method.

So what about the schema? So far, we haven’t seen any evidence that Ming is doing anything with the schema information at all. Well, the first way that Ming helps us is by making sure we don’t specify values for properties that are not defined in the object:

>>> page = tutorial.WikiPage(dict(title='MyPage', text='', fooBar=''))
>>> page
{'fooBar': '', 'text': '', 'title': 'MyPage'}
>>> page.m.save()
Traceback (most recent call last):
  ...
formencode.api.Invalid: <class 'tutorial.WikiPage'>:
    Extra keys: set(['fooBar'])

OK, that’s nice and all, but wouldn’t it be nicer if we could be warned at creation time? Ming provides a convenice method make() on the ming.Document with just such behavior:

>>> page = tutorial.WikiPage.make(dict(title='MyPage', text='', fooBar=''))
Traceback (most recent call last):
  ...
formencode.api.Invalid: <class 'tutorial.WikiPage'>:
    Extra keys: set(['fooBar'])

We can also provide default values for properties via the if_missing parameter on a Field. Change the definition of the text property in tutorial.py to read:

text = Field(str, if_missing='')

Now if we restart the interpreter (or reload the tutorial module), we can do the following:

>>> page = tutorial.WikiPage.make(dict(title='MyPage'))
>>> page
{'text': '', 'title': 'MyPage'}

Ming also supports supplying a callable as an if_missing value so you could put the creation date in a WikiPage like this:

from datetime import datetime

...

creation_date = Field(datetime, if_missing=datetime.utcnow)

API Reference

This include reference to all the Ming classes and methods.

Contents:

ming module

ming.configure(**kwargs)

Given a (flat) dictionary of config values, creates DataStores and saves them by name

class ming.Index(*fields, **kwargs)

Represents a MongoDB Index of a field.

ming.base module

Ming Base module. Good stuff here.

ming.base.Missing = <Missing>

This is the value that Missing fields in MongoDB documents receive.

Functions

ming.base._safe_bson(obj, _no_warning=False)

Verify that the obj is safe for bsonification (in particular, no tuples or Decimal objects

Classes

class ming.base.Cursor(cls, cursor, allow_extra=True, strip_extra=True)

Bases: object

Python class proxying a MongoDB cursor, constructing and validating objects that it tracks

all()

count()

distinct(*args, **kwargs)

first()

hint(index_or_name)

limit(limit)

next()

one()

rewind()

skip(skip)

sort(*args, **kwargs)

class ming.metadata._Document(data=None, skip_from_bson=False)

Bases: ming.base.Object

classmethod make(data, **kwargs)

Kind of a virtual constructor

class ming.declarative._DocumentMeta

Bases: type

mro()

Return a type’s method resolution order.

class ming.metadata.Field(*args, **kwargs)

Bases: object

Represents a Field in a MongoDB Document

It accepts one or two positional arguments and any number of keyword arguments.

If only one positional argument is provided that is considered to be the schema of the Field which is one of the types available in ming.schema module.

If two positional arguments are provided the first is considered being the name of the field inside the MongoDB Document and the second is still the schema type.

The index, unique and sparse keyword arguments are also consumed by Field to automatically create an index for the field.

Additional arguments are just forwarded to Schema constructor if the provided schema argument is not already an instance. So see the classes declared in ming.schema for details on the other available keyword arguments.

class ming.metadata._ClassManager(cls, collection_name, session, fields, indexes, polymorphic_on=None, polymorphic_identity=None, polymorphic_registry=None, version_of=None, migrate=None, before_save=None)

Bases: object

InstanceManagerClass

alias of _InstanceManager

add_index(idx)

aggregate(*args, **kwargs)

before_save = None

collection

count(*args, **kwargs)

distinct(*args, **kwargs)

drop_indexes(*args, **kwargs)

ensure_index(*args, **kwargs)

ensure_indexes(*args, **kwargs)

fields

find(*args, **kwargs)

find_and_modify(*args, **kwargs)

find_by(*args, **kwargs)

get(*args, **kwargs)

group(*args, **kwargs)

index_information(*args, **kwargs)

inline_map_reduce(*args, **kwargs)

make(data, allow_extra=False, strip_extra=True)

map_reduce(*args, **kwargs)

migrate()

Load each doc in the collection and immediately save it

polymorphic_on

polymorphic_registry

remove(*args, **kwargs)

update_partial(*args, **kwargs)

with_session(session)

Return a Manager with an alternate session

class ming.metadata._InstanceManager(mgr, inst)

Bases: object

delete(*args, **kwargs)

increase_field(*args, **kwargs)

insert(*args, **kwargs)

save(*args, **kwargs)

set(*args, **kwargs)

upsert(*args, **kwargs)

class ming.metadata._ManagerDescriptor(manager)

Bases: object

engine

class ming.base.Object

Bases: dict

Dict providing object-like attr access

classmethod from_bson(bson)

make_safe()

ming.datastore module

class ming.datastore.DataStore(bind, name, authenticate=None)

Represents a Database on a specific MongoDB Instance.

DataStore keeps track of a specific database on a MongoDB Instance, Cluster or ReplicaSet. The database is represented by its name while MongoDB is represented by an Engine instance.

DataStores are usually created using the create_datastore() function.

db

This is the database on MongoDB.

Accessing this property returns the pymongo db, untracked by Ming.

class ming.datastore.Engine(Connection, conn_args, conn_kwargs, connect_retry, auto_ensure_indexes, _sleep=<built-in function sleep>)

Engine represents the connection to a MongoDB (or in-memory database).

The Engine class lazily creates the connection the firs time it’s actually accessed.

conn

This is the pymongo connection itself.

connect()

Actually opens the connection to MongoDB.

This is usually done automatically when accessing a database for the first time through the engine.

ming.datastore.create_datastore(uri, **kwargs)

Creates a new DataStore for the database identified by uri.

uri is a mongodb url in the form mongodb://username:password@address:port/dbname, it can also be used to connect to a replica set by specifying multiple mongodb addresses separated by comma set mongodb://localhost:27018,localhost:27019,localhost:27020/dbname?replicaSet=rsname.

Optional Keyword args:

• bind - a ming.datastore.Engine instance

All other keyword args are passed along to create_engine().

The following are equivalent:

• create_datastore(‘mongodb://localhost:27017/foo’)
• create_datastore(‘foo’)
• create_datastore(‘foo’, bind=create_engine())

ming.odm module

class ming.odm.declarative.MappedClass

Declares a Ming Mapped Document.

Mapped Documents provide a declarative interface to schema, relations and properties declaration for your Models stored as MongoDB Documents.

MappedClasses required that a __mongometa__ subclass is available inside which provides the details regarding the name of the collection storing the documents and the session used to store the documents:

class WikiPage(MappedClass):
    class __mongometa__:
        session = session
        name = 'wiki_page'

    _id = FieldProperty(schema.ObjectId)
    title = FieldProperty(schema.String(required=True))
    text = FieldProperty(schema.String(if_missing=''))

ming.odm.base.session(v) → ODMSession

Returns the ORMSession instance managing either a class or an object

ming.odm.base.state(obj) → ming.odm.base.ObjectState

Gets the UnitOfWork state of a mapped object

exception ming.odm.property.AmbiguousJoin

class ming.odm.property.FieldProperty(field_type, *args, **kwargs)

Declares property for a value stored in a MongoDB Document.

The provided arguments are just forwarded to Field class which is actually in charge of managing the value and its validation.

For details on available options in FieldProperty just rely on Field documentation.

include_in_repr

bool(x) -> bool

Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

class ming.odm.property.FieldPropertyWithMissingNone(field_type, *args, **kwargs)

A class like FieldProperty with one exception. If you use if_missing=S.Missing with FieldPropertyWithMissingNone when a value in Mongo is not present, instead of Ming throwing an AttributeError, Ming will return a value of None for that attribute.

class ming.odm.property.ForeignIdProperty(related, uselist=False, allow_none=False, *args, **kwargs)

Declares a field to store one or more ObjectIds of a related objects.

The related argument must be the related entity class or the name of the related entity class (to avoid circular dependencies). The field itself will actually store and retrieve bson.ObjectId instances.

uselist argument can be used to tell Ming whenever the object can relate to more than one remote object (many-to-many) and so the ids are stored in a MongoDB Array.

Usually a ForeignIdProperty with value None means that the object is not related to any other entity, in case you have entities that might have None as their ids you can use allow_none option to tell Ming that None is a valid foreign id.

exception ming.odm.property.NoJoin

exception ming.odm.property.ORMError

class ming.odm.property.RelationProperty(related, via=None, fetch=True)

Provides a way to access OneToMany, ManyToOne and ManyToMany relations.

The RelationProperty relies on ForeignIdProperty to actually understand how the relation is composed and how to retrieve the related data.

Assigning a new value to the relation will properly update the related objects.

class ming.odm.odmsession.ODMCursor(session, cls, ming_cursor, refresh=False, decorate=None, fields=None)

Represents the results of query.

The cursors can be iterated over to retrieve the results one by one.

all()

Retrieve all the results of the query

count()

Get the number of objects retrieved by the query

first()

Gets the first result of the query

limit(limit)

Limit the number of entries retrieved by the query

one()

Gets one result and exaclty one.

Raises ValueError exception if less or more than one result is returned by the query.

rewind()

Rewind this cursor to its unevaluated state. Reset this cursor if it has been partially or completely evaluated. Any options that are present on the cursor will remain in effect. Future iterating performed on this cursor will cause new queries to be sent to the server, even if the resultant data has already been retrieved by this cursor.

skip(skip)

Skip the first skip entries retrieved by the query

sort(*args, **kwargs)

Sort results of the query.

See pymongo.cursor.Cursor.sort() for details on the available arguments.

class ming.odm.odmsession.ODMSession(doc_session: ming.session.Session = None, bind: ming.datastore.DataStore = None, extensions=None, autoflush=False)

Current Ming Session.

Keeps track of active objects in the IdentityMap and UnitOfWork and of the current connection to the database. All the operation on MongoDB should happen through the ODMSession to avoid inconsistent state between objects updated through the session and outside the session.

aggregate(cls, *args, **kwargs)

Runs an aggregation pipeline on the given collection.

Arguments are the same as pymongo.collection.Collection.aggregate().

classmethod by_name(name)

Retrieve or create a new Session with the given name.

This is useful to keep around multiple sessions and identify them by name. The session registry is global so they are available everywhere as far as the ming module is the same.

clear()

Expunge all the objects from the session.

close()

Clear the session.

db

Access the low-level pymongo database

distinct(cls, *args, **kwargs)

Get a list of distinct values for a key among all documents in this collection.

Arguments are the same as pymongo.collection.Collection.distinct().

drop_indexes(cls)

Drop all indexes declared in cls

ensure_indexes(cls)

Ensures all indexes declared in cls

expunge(obj)

Remove an object from the Session (and its UnitOfWork and IdentityMap)

find(cls, *args, **kwargs)

Retrieves cls by performing a mongodb query.

This is the same as calling cls.query.find() and always performs a query on the database. According to the refresh argument the objects are also updated in the UnitOfWork or not. Otherwise the UnitOfWork keeps the old object state which is the default.

If the session has autoflush option, the session if flushed before performing the query.

Arguments are the same as pymongo.collection.Collection.find() plus the following additional arguments:

• allow_extra Whenever to raise an exception in case of extra fields not specified in the model definition.
• strip_extra Whenever extra fields should be stripped if present.
• validate Disable validation or not.

It returns an ODMCursor with the results.

find_and_modify(cls, *args, **kwargs)

Finds and updates cls.

Arguments are the same as pymongo.collection.Collection.find_and_modify().

If the session has autoflush option, the session if flushed before performing the query.

It returns an ODMCursor with the results.

flush(*args, **kwargs)

Flush obj or all the objects in the UnitOfWork.

When obj is provided, only obj is flushed to the database, otherwise all the objects in the UnitOfWork are persisted on the databases according to their current state.

get(cls, idvalue)

Retrieves cls by its _id value passed as idvalue.

If the object is already available in the IdentityMap this acts as a simple cache a returns the current object without querying the database. Otherwise a find query is issued and the object retrieved.

This the same as calling cls.query.get(_id=idvalue).

group(cls, *args, **kwargs)

Runs a grouping on the model collection.

Arguments are the same as pymongo.collection.Collection.group().

inline_map_reduce(cls, *args, **kwargs)

Runs a MapReduce job and keeps results in-memory.

Arguments are the same as pymongo.collection.Collection.inline_map_reduce().

map_reduce(cls, *args, **kwargs)

Runs a MapReduce job and stores results in a collection.

Arguments are the same as pymongo.collection.Collection.map_reduce().

refresh(obj)

Refreshes the object in the session by querying it back and updating its state

remove(*args, **kwargs)

Delete one or more cls entries from the collection.

This performs the delete operation outside of the UnitOfWork, so the objects already in the UnitOfWork and IdentityMap are unaffected and might be recreated when the session is flushed.

Arguments are the same as pymongo.collection.Collection.remove().

save(obj)

Add an object to the Session (and its UnitOfWork and IdentityMap).

Usually objects are automatically added to the session when created. This is done by _InitDecorator.saving_init() which is the __init__ method of all the :class`.MappedClass` subclasses instrumented by Mapper.

So calling this method is usually not required unless the object was expunged.

update(cls, spec, fields, **kwargs)

Updates one or more cls entries from the collection.

This performs the update operation outside of the UnitOfWork, so the objects already in the UnitOfWork and IdentityMap are unaffected and might be restored to previous state when the session is flushed.

Arguments are the same as pymongo.collection.Collection.update().

update_if_not_modified(obj, fields, upsert=False)

Updates one entry unless it was modified since first queried.

Arguments are the same as pymongo.collection.Collection.update().

Returns whenever the update was performed or not.

class ming.odm.odmsession.SessionExtension(session)

Base class that should be inherited to handle Session events.

after_cursor_next(cursor)

Cursor has advanced to next result

after_delete(obj, st)

After an object gets deleted in this session

after_flush(obj=None)

After the session is flushed for obj

If obj is None it means all the objects in the UnitOfWork which can be retrieved by iterating over ODMSession.uow

after_insert(obj, st)

After an object gets inserted in this session

after_remove(cls, *args, **kwargs)

After a remove query is performed session

after_update(obj, st)

After an object gets updated in this session

before_cursor_next(cursor)

Cursor is going to advance to next result

before_delete(obj, st)

Before an object gets deleted in this session

before_flush(obj=None)

Before the session is flushed for obj

If obj is None it means all the objects in the UnitOfWork which can be retrieved by iterating over ODMSession.uow

before_insert(obj, st)

Before an object gets inserted in this session

before_remove(cls, *args, **kwargs)

Before a remove query is performed session

before_update(obj, st)

Before an object gets updated in this session

cursor_created(cursor, action, *args, **kw)

New cursor with the results of a query got created

class ming.odm.odmsession.ThreadLocalODMSession(*args, **kwargs)

ThreadLocalODMSession is a thread-safe proxy to ODMSession.

This routes properties and methods to the session in charge of the current thread. For a reference of available methods and properties refer to the ODMSession.

classmethod by_name(name)

Retrieve or create a new ThreadLocalODMSession with the given name.

This is useful to keep around multiple sessions and identify them by name. The session registry is global so they are available everywhere as far as the ming module is the same.

classmethod close_all()

Closes all the ODMSessions registered in current thread, which clears their objects tracked in memory.

Does not close connections.

classmethod flush_all()

Flush all the ODMSessions registered in current thread

Usually is not necessary as only one session is registered per-thread.

class ming.odm.mapper.Mapper(mapped_class, collection, session, **kwargs)

Keeps track of MappedClass subclasses.

The Mapper is in charge of linking together the Ming Schema Validation layer, the Session and a MappedClass. It also compiles the Schema Validation for Mapped Classes if they don’t already have one.

Mapper also instruments mapped classes by adding the additional .query property and behaviours.

You usually won’t be using the Mapper directly apart from compile_all() and ensure_all_indexes() methods.

classmethod compile_all()

Compiles Schema Validation details for all MappedClass subclasses

classmethod ensure_all_indexes()

Ensures indexes for each registered MappedClass subclass are created

class ming.odm.mapper.MapperExtension(mapper)

Base class that should be inherited to handle Mapper events.

after_delete(instance, state, sess)

Receive an object instance and its current state after that instance is deleted.

after_insert(instance, state, sess)

Receive an object instance and its current state after that instance is inserted into its collection.

after_remove(sess, *args, **kwargs)

After a remove query is performed for this class

after_update(instance, state, sess)

Receive an object instance and its current state after that instance is updated.

before_delete(instance, state, sess)

Receive an object instance and its current state before that instance is deleted.

before_insert(instance, state, sess)

Receive an object instance and its current state before that instance is inserted into its collection.

before_remove(sess, *args, **kwargs)

Before a remove query is performed for this class

before_update(instance, state, sess)

Receive an object instance and its current state before that instance is updated.

ming.odm.mapper.mapper(cls, collection=None, session=None, **kwargs)

Gets or creates the mapper for the given cls MappedClass

class ming.odm.middleware.MingMiddleware(app, context_func=None, flush_on_errors=())

WSGI Middleware that automatically flushes and closes ODM Sessions.

At the end of each WSGI request for app the middleware will automatically flush the session unless there was an Exception, then it will close the session to clear it.

flush_on_errors can be a list of exception types for which the session should be flushed anyway. This usually includes webob.exc.HTTPRedirection subclasses if WebOb is available.

ming.schema module

class ming.schema.Anything(required=<NoDefault>, if_missing=<NoDefault>)

Accepts any value without validation.

Passes validation unchanged except for converting dict => Object

This is often used to avoid the cost of validation on fields that might contain huge entries that do not need to be validated.

Parameters:

• required (bool) – if True and this field is missing, an Invalid exception will be raised
• if_missing (value or callable) – provides a default value for this field if the field is missing

validate(value, **kw)

Validate an object or raise an Invalid exception.

Default implementation just raises NotImplementedError

class ming.schema.Array(field_type, **kw)

Validates a MongoDB Array.

All elements of the array must pass validation by a single field_type (which itself may be Anything, however).

class ming.schema.Binary(**kw)

Validates value is a bson.Binary

class ming.schema.Bool(**kw)

Validates value is a bool

type

alias of builtins.bool

class ming.schema.DateTime(**kw)

Validates value is a datetime and ensures its on UTC.

If value is a datetime but it’s not on UTC it gets converted to UTC timezone. if value is instance of date it will be converted to datetime

_validate(value, **kw)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

class ming.schema.DateTimeTZ(**kw)

Validates value is a datetime.

type

alias of datetime.datetime

class ming.schema.Deprecated

Used for deprecated fields – they will be stripped from the object.

validate(value, **kw)

Validate an object or raise an Invalid exception.

Default implementation just raises NotImplementedError

class ming.schema.Document(fields=None, required=False, if_missing=<NoDefault>)

Specializes Object adding polymorphic validation.

This is usually not used directly, but it’s the Schema against which each document in Ming is validated to.

Polymorphic Validation means that Document._validate(…) sometimes will return an instance of ChildClass based on .polymorphic_on entry of the object (its value is used to determine the class it should be promoted based on polymorphic_identity of .managed_class).

Map of polymorphic_identity values are stored in .polymorphic_registry dictionary.

By default .polymorphic_on, .managed_class and .polymorphic_registry are all None. So .managed_class must be set and and set_polymorphic() method must be called to configure them properly.

_validate(d, allow_extra=False, strip_extra=False)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

get_polymorphic_cls(data)

Given a mongodb document it returns the class it should be converted to.

set_polymorphic(field, registry, identity)

Configure polymorphic behaviour (except for .managed_class).

field is the .polymorphic_on, which is the name of the field used to detect polymorphic type.

registry is the .polymorphic_registry which is a map of field values to classes.

identity is the value which leads to .managed_class type itself.

validate(value, **kw)

Validate an object or raise an Invalid exception.

Default implementation just raises NotImplementedError

class ming.schema.FancySchemaItem(required=<NoDefault>, if_missing=<NoDefault>)

SchemaItem that provides support for required fields and default values.

If the value is present, then the result of _validate() method is returned.

Parameters:

• required (bool) – if True and this field is missing, an Invalid exception will be raised
• if_missing (value or callable) – provides a default value for this field if the field is missing

_validate(value, **kw)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

class ming.schema.Float(**kw)

Validates value is int or float

class ming.schema.Int(**kw)

Validates value is an int.

Also accepts float which represent integer numbers like 10.0 -> 10.

_validate(value, **kw)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

exception ming.schema.Invalid(msg, value, state=None, error_list=None, error_dict=None)

This is raised in response to invalid input. It has several public attributes:

msg:

The message, without values substituted. For instance, if you want HTML quoting of values, you can apply that.

substituteArgs:

The arguments (a dictionary) to go with msg.

str(self):

The message describing the error, with values substituted.

value:

The offending (invalid) value.

state:

The state that went with this validator. This is an application-specific object.

error_list:

If this was a compound validator that takes a repeating value, and sub-validator(s) had errors, then this is a list of those exceptions. The list will be the same length as the number of values – valid values will have None instead of an exception.

error_dict:

Like error_list, but for dictionary compound validators.

class ming.schema.Migrate(old, new, migration_function)

Use when migrating from one Schema to another.

The validate() method first tries to apply the new SchemaItem and if it fails tries the old SchemaItem. If the new fails but the old succeeds the migration_function will be called to upgrade the value from the old schema to the new one.

This is also used by Ming to perform migrations on whole documents through the version_of attribute.

classmethod obj_to_list(key_name, value_name=None)

Factory Method that creates migration functions to convert a dictionary to list of objects.

Migration function will go from object { key: value } to list [ { key_name: key, value_name: value} ].

If value_name is None, then value must be an object which will be expanded in the resulting object itself: [ { key_name: key, **value } ].

validate(value, **kw)

First tries validation against new and if it fails applies migrate_function.

The migrate_function is applied only if old validation succeeds, this is to ensure that we are not actually facing a corrupted value.

If old validation fails, the method just raises the validation error.

class ming.schema.NumberDecimal(precision=None, rounding='ROUND_HALF_DOWN', **kwargs)

Validates value is compatible with bson.Decimal128.

Useful for financial data that need arbitrary precision.

The argument precision specifies the number of decimal digits to evaluate. Defaults to 6.

The argument rounding specifies the kind of rounding to be performed. Valid values are decimal.ROUND_DOWN, decimal.ROUND_HALF_UP, decimal.ROUND_HALF_EVEN, decimal.ROUND_CEILING, decimal.ROUND_FLOOR, decimal.ROUND_UP, decimal.ROUND_HALF_DOWN, decimal.ROUND_05UP. Defaults to decimal.ROUND_HALF_DOWN.

_validate(value, **kw)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

class ming.schema.Object(fields=None, required=False, if_missing=<NoDefault>)

Used for dict-like validation.

It validates all the values inside the dictionary and converts it to a ming.base.Object.

Also ensures that the incoming object does not have any extra keys.

_validate(d, allow_extra=False, strip_extra=False)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

if_missing()

Missing is a sentinel used to indicate a missing key or missing keyword argument (used since None sometimes has meaning)

class ming.schema.ObjectId(required=<NoDefault>, if_missing=<NoDefault>)

Validates value is a bson.ObjectId.

If value is a string that represents an ObjectId it gets converted to an ObjectId.

If the ObjectId is not provided (or it’s Missing) a new one gets generated. To override this behaviour pass if_missing=None to the schema constructor:

>>> schema.ObjectId().validate(schema.Missing)
ObjectId('55d5df957ab71c0a5c3647bd')
>>> schema.ObjectId(if_missing=None).validate(schema.Missing)
None

Parameters:

• required (bool) – if True and this field is missing, an Invalid exception will be raised
• if_missing (value or callable) – provides a default value for this field if the field is missing

_validate(value, **kw)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

if_missing()

Provides a bson.ObjectId as default

class ming.schema.OneOf(*options, **kwargs)

Expects validated value to be one of options.

This is often used to validate against Enums.

_validate(value, **kw)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

class ming.schema.ParticularScalar(**kw)

Specializes Scalar to also check for a specific type.

This is usually not directly used, but its what other validators rely on to implement their behaviour.

_validate(value, **kw)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

type = ()

Expected Type of the validated value.

class ming.schema.Scalar(required=<NoDefault>, if_missing=<NoDefault>)

Validate that a value is NOT an array or dict.

This is used to validate single values in MongoDB Documents.

Parameters:

• required (bool) – if True and this field is missing, an Invalid exception will be raised
• if_missing (value or callable) – provides a default value for this field if the field is missing

_validate(value, **kw)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

class ming.schema.SchemaItem

Basic Schema enforcement validation.

The validate() method is called when a record is loaded from the DB or saved to it.

It should return a “validated” object, raising a Invalid exception if the object is invalid. If it returns Missing, the field will be stripped from its parent object.

This is just the base class for schema validation. When implementing your own schema validation for a Ming Field you probably want to subclass FancySchemaItem.

classmethod make(field, *args, **kwargs)

Factory method for schemas based on a python type.

This accepts a python type as its argument and creates the appropriate SchemaItem that validates it.

Accepted arguments are:

• int - int or long
• str - string or unicode
• float - float, int, or long
• bool - boolean value
• datetime - datetime.datetime object
• None - Anything
• [] - Array of Anything objects
• [type] - array of objects of type “type”
• { fld: type… } - dict-like object with field “fld” of type “type”
• { type: type… } - dict-like object with fields of type “type”
• anything else (e.g. literal values), must match exactly

*args and **kwargs are passed on to the specific class of SchemaItem created.

validate(d, **kw)

Validate an object or raise an Invalid exception.

Default implementation just raises NotImplementedError

class ming.schema.String(**kw)

Validates value is str or unicode string

class ming.schema.Value(value, **kw)

Checks that validated value is exactly value

_validate(value, **kw)

Performs actual validation.

Subclasses are expected to override this method to provide the actual validation.

The default implementation leaves the value as is.

ming.utils module

Functions

ming.utils.encode_keys(d)

Encodes the unicode keys of d, making the result a valid kwargs argument

ming.utils.all_class_properties(cls)

Find all properties of the class, including those inherited

Classes

class ming.utils.LazyProperty(func)

Bases: object

Ming News / Release Notes

The latest releases support PyMongo 3. The older 0.5.x releases support PyMongo 2 and Python 2.

0.13.0 (Mar 16, 2023)

• remove Python 3.6 support
• set all DeprecationWarning’s stacklevel=2 to show caller
• MIM: verify kwargs in find/find_one

0.12.2 (Nov 15, 2022)

• MIM: add support for UUID types
• improve type hints

0.12.1 (Sep 13, 2022)

• allow Field(bytes) to work like Field(S.Binary)
• handle rare race condition exception
• MIM: support cursor/find as context manager
• MIM: handle bytes & Binary in queries
• MIM: handle queries given as RawBSONDocument
• improve type hints
• run tests on 3.10 and 3.11
• test fix for python 3.11
• test suite can be run in parallel (tox -p auto)

0.12.0 (Jun 2, 2022)

• Remove support for python < 3.6

0.11.2 (Oct 15, 2021)

• MIM: support distinct() usage on fields that are lists
• improve a few type hints

0.11.1 (Sep 9, 2021)

• Include py.typed and .pyi files in distribution

0.11.0 (Sep 9, 2021)

• Drop support for Python 2.7, 3.3, and 3.4
• Support for Python 3.9
• MIM: support sparse unique indexes
• propagate return values from various update/delete/insert operations
• Support __init_subclass__ arguments
• validate() may not have been validating sub-documents
• Add some type annotations

0.10.2 (Jun 19, 2020)

• Fix error using save() and no _id
• MIM: Avoid errors from _ensure_orig_key when positional $ is used

0.10.1 (Jun 17, 2020)

• fix situation with gridfs indexes and MIM
• fix validate=False and update some MIM params to match pymongo closer

0.10.0 (Jun 8, 2020)

• Support for PyMongo 3.10
• Support for Python 3.8
• Removed start_request/end_request from MIM
• Added Cursor.close to MIM
• Moved testing from nose to unittest

0.9.2 (Mar 12, 2020)

• Support ODM before_save hook on Python 3

0.9.1 (May 15, 2019)

• Allow usage of PyMongo 3.7

0.9.0 (Feb 22, 2019)

• Support for Decimal128 type in MongoDB through schema.NumberDecimal
• Deprecation of make_safe for Ming Documents

0.8.1 (Feb 22, 2019)

• Fix for connection string when seedlist is used

0.8.0 (Jan 15, 2019)

• FormEncode is now an optional dependency only required for projects relying on ming.configure.
• Python 3.7 is now officially supported

0.7.1 (Nov 30, 2018)

• MIM: Minimal support for Collection.aggregate

0.7.0 (May 10, 2018)

• MIM: Support for PyMongo 3.6
• MIM: Partial support for $text queries
• MIM: Make created index match more the style pymongo itself stores indexes.
• MIM: Support matching $regex against arrays.
• MIM: Support fake $score in projections.
• MIM: Support $slice in projections.
• MIM: Partial support for bulk writes, currently only UpdateOne.

0.5.7 (Mar 12, 2020)

• Support ODM before_save hook on Python 3

0.5.6 (Apr 2, 2018)

• MIM: match correctly when search values are lists or dicts more than 1 level deep.

0.6.1 (Sep 27, 2017)

• MIM: Support searching for $regex that contain text instead of only “startswith”

0.6.0 (Sep 24, 2017)

• Support new PyMongo 3.X API
• MIM: Fix duplicated keys are detected on upsertions
• MIM: Support for filters on distinct
• MIM: Provide drop_indexes
• MIM: Simulate collstats
• MIM: Support insert_one and insert_many results
• MIM: Support update_one and update_many results
• MIM: Support indexing whole subdocuments
• MIM: Support for setOnInsert

0.5.5 (Nov 30, 2016)

• MIM: do not raise errors when regex matching against None or non-existant values

0.5.4 (Apr 29, 2016)

• On Python3, bson.Binary actually decodes to bytes
• Support distinct() on cursors (already supported on collections)

0.5.3 (Oct 18, 2015)

• Documentation Rewrite
• Speed improvements on ODM queries that retrieve objects not already tracked by UOW/IMAP.
• Mapper now provides .ensure_all_indexes() method to ensure indexes for all registered mappers.
• MappedClass (ODM Declarative) now supports version_of and migrate for migrations.
• MappedClass.query.get now supports _id as its first positional argument
• ODMSession constructor now exposes the autoflush argument to automatically flush session before ODM queries, previously it was always forced to False. Pay attention that as MongoDB has no transactions autoflush will actually write the changes to the database.
• ODMSession now exposes .db and .bind properties which lead to the underlying pymongo database and DataStore
• Fixed ODMSession.by_name which previously passed the datastore as session argument.
• ODMSession now provides .refresh method that updates a specific object from the database
• ThreadLocalODMSession now provides by_name method to configure Thread Safe sessions using ming.configure
• ming.schema.Invalid now has default None argument for state, it was never used by the way.

0.5.2 (Apr 16, 2015)

• Support for text indexes
• Specify our requirement on pymongo < 3.0 (until supported)

0.5.1 (Apr 6, 2015)

• Cursor compatibility for Python 3

0.5.0 (Jun 5, 2014)

• Compatible with pymongo 2.7
• Compatible with Python 3.3 and 3.4
• Compatible with PyPy
• Fix update_if_not_modified
• MIM: support float comparisons
• ming.configure now allows any extra params to pass through to MongoClient

0.4.7 (Apr 16, 2014)

• Add allow_none option to ForeignIdProperty

0.4.6 (Apr 4, 2014)

• Fixed issue with if_missing for ForeignIdProperty

0.4.5 (Apr 4, 2014)

• avoid extremely long error text
• Fixed random generated ObjectId on empty ForeignIdProperty

0.4.4 (Mar 10, 2014)

• Revert ForeignIdProperty None optimization
• Fix delete event hook signatures
• Fix typo when flushing an individual object flagged for deletion

0.4.3 (Jan 7, 2014)

• Return result of update_partial()
• ManyToMany support relying on a list of ObjectIds
• Make RelationProperty writable
• Support for all pymongo options in custom_indexes declaration
• Permit relationships that point to same model
• Fix wrong behavior for MIM find_and_modify new option and add test case
• ForeignIdProperty None optimization

0.4.2 (Sep 26, 2013)

• bool(cursor) now raises an Exception. Pre-0.4 it evaluated based on the value of __len__ but since 0.4 removed __len__ it always returned True (python’s default behavior) which could be misleading and unexpected. This forces application code to be changed to perform correctly.
• schema migration now raises the new schema error if both old & new are invalid
• aggregation methods added to session. distinct, aggregate, etc are now available for convenience and pass through directly to pymongo
• MIM: support for indexing multi-valued properties
• MIM: forcing numerical keys as strings
• MIM: add manipulate arg to insert for closer pymongo compatibility

0.4.1 and 0.3.9 (Aug 30, 2013)

• MIM: Support slicing cursors
• MIM: Fixed exact dot-notation queries
• MIM: Fixed dot-notation queries against null fields
• MIM: Translate time-zone aware timestamps to UTC timestamps. pytz added as dependency
• MIM: Allow the remove argument to find_and_modify

0.4 (June 28, 2013)

• removed ‘flyway’ package from ming. It is now available from https://github.com/amol-/ming-flyway This removes the dependency on PasteScript and will make Python 3 migration easier.
• WebOb dependency is optional.
• removed cursor.__len__ You must change len(query) to query.count() now. This prevents inadvertent extra count queries from running. https://sourceforge.net/p/merciless/bugs/18/

0.3.2 through 0.3.8

• many improvements to make MIM more like actual mongo
• various fixes and improvements

0.3.2 (rc1) (January 8, 2013)

Some of the larger changes:

• Update to use MongoClient everywhere instead of variants of pymongo.Connection
• Remove MasterSlaveConnection and ReplicaSetConnection support

0.3.2 (dev) (July 26, 2012)

Whoops, skipped a version there. Anyway, the bigger changes:

• Speed improvements in validation, particularly validate_ranges which allows selective validation of arrays
• Allow requiring scalar values to be non-None
• Add support for geospatial indexing
• Updates to engine/datastore creation syntax (use the new create_engine or create_datastore, which are significantly simplified and improved).

0.3 (March 6, 2012)

Lots of snapshot releases, and finally a backwards-breaking change. The biggest change is the renaming of the ORM to be the ODM.

• Renamed ming.orm to ming.odm
• Lots of bug fixes
• Add gridfs support to Ming
• Add contextual ODM session

0.2.1

It’s been a lonnnnng time since our last real release, so here are the high points (roughly organized from low-level to high-level):

• Support for replica sets
• Support for using gevent with Ming (asynchronous Python library using libevent)
• Add find_and_modify support
• Create Mongo-in-Memory support for testing (mim:// url)
• Some don’t shoot-yourself-in-the-foot support (calling .remove() on an instance, for example)
• Move away from using formencode.Invalid exception
• Allow skipping Ming validation, unsafe inserts
• Elaborate both the imperative and declarative support in the document- and ORM-layers
• Polymorphic inheritance support in the ORM

Indices and tables

• Index
• Module Index
• Search Page