Merge remote-tracking branch 'origin/master' into use-django-uuid-field

.travis.yml

@@ -2,16 +2,8 @@ language: python
 python: "2.7"
 
 env:
-  - TOX_ENV=py27-django16-pg
-  - TOX_ENV=py27-django16-sqlite
-  - TOX_ENV=py27-django17-pg
-  - TOX_ENV=py27-django17-sqlite
   - TOX_ENV=py27-django18-pg
   - TOX_ENV=py27-django18-sqlite
-  - TOX_ENV=py34-django16-pg
-  - TOX_ENV=py34-django16-sqlite
-  - TOX_ENV=py34-django17-pg
-  - TOX_ENV=py34-django17-sqlite
   - TOX_ENV=py34-django18-pg
   - TOX_ENV=py34-django18-sqlite
 

NOTICE

@@ -4,6 +4,6 @@ Copyright 2014 Swisscom, Sophia Engineering
 This software contains code derived from DirtyVersion available at
 https://github.com/cordmata/dirtyversion
 
-This software was tested on Django 1.6 & 1.7 (https://www.djangoproject.com/)
+This software was tested on Django 1.8 (https://www.djangoproject.com/)
 
-This software is written in Python 2.7 (https://www.python.org/)
+This software is written for Python 2.7 and Python 3.4 (https://www.python.org/)

README.rst

@@ -20,7 +20,6 @@ relational database. It allows to keep track of modifications on an object over
 
 CleanerVersion therefore enables a Django-based Datawarehouse, which was the initial idea of this package.
 
-
 Features
 ========
 
@@ -50,9 +49,17 @@ Prerequisites
 This code was tested with the following technical components
 
 * Python 2.7 & 3.4
-* Django 1.6, 1.7 & 1.8
+* Django 1.8
 * PostgreSQL 9.3.4 & SQLite3
 
+Older Django versions
+=====================
+CleanerVersion was originally written for Django 1.6.
+
+Old packages compatible with older Django releases:
+
+* Django 1.6 and 1.7: https://pypi.python.org/pypi/CleanerVersion/1.5.4
+
 
 Documentation
 =============

docs/doc/historization_with_cleanerversion.rst

@@ -3,7 +3,7 @@ Historization with CleanerVersion
 *********************************
 
 Disclaimer: This documentation as well as the CleanerVersion application code have been written to work against Django
-1.6.x, 1.7.x and 1.8.x. The documentation may not be accurate anymore when using more recent versions of Django.
+1.8.x. The documentation may not be accurate anymore when using more recent versions of Django.
 
 .. _cleanerversion-quick-starter:
 
@@ -323,12 +323,10 @@ Let's create two disciplines and some sportsclubs practicing these disciplines::
     lca = SportsClub.objects.create(name='LCA', practice_periodicity='individual',
                                                 discipline=running)
 
+    t1 = datetime.utcnow().replace(tzinfo=utc)
+
 Reading objects from a M2O relationship
 ---------------------------------------
-
-Assume, timestamps have been created as follows::
-    timestamp = datetime.datetime.utcnow().replace(tzinfo=utc)
-
 Now, let's read some stuff previously loaded::
 
     sportsclubs = SportsClub.objects.as_of(t1)  # This returns all SportsClubs existing at time t1 [returned within a QuerySet]
@@ -349,7 +347,87 @@ Note that select_related only works for models containing foreign keys.  It does
 
 This is not a CleanerVersion limitation; it's just the way that Django's select_related() works.  Use
 prefetch_related() instead if you want to prefetch reverse or many-to-many relationships.  Note that
-prefetch_related() will use at least two queries to prefetch the related objects.
+prefetch_related() will use at least two queries to prefetch the related objects.  See also
+the :ref:`prefetch_related_notes`.
+
+
+Filtering using objects
+^^^^^^^^^^^^^^^^^^^^^^^
+Following on the above example, let's create a new version of the running Discipline.  First, though, let's take
+a look at the id, identities and foreign keys as they are now::
+
+    >> (running.id, running.identity)
+    (1, 1)
+    >> (stb.discipline_id, stb.id, stb.identity)
+    (1, 10, 10)
+    >> (lca.discipline_id, lca.id, lca.identity)
+    (1, 20, 20)
+
+OK, so now we create a new version::
+
+    running = running.clone()
+    running.rules = "Don't run on other's feet"
+    running.save()
+
+    # Fetch the old version from the database:
+    running_at_t1 = Discipline.objects.as_of(t1).get(name='Running')
+
+How do the id, identities, and foreign keys look at this point?
+::
+
+    >> (running.id, running.identity)
+    (1, 1)
+
+    >> (running_at_t1.id, running_at_t1.identity)
+    (2, 1)
+
+    >> (stb.discipline_id, stb.id, stb.identity)
+    (1, 10, 10)
+
+    >> (lca.discipline_id, lca.id, lca.identity)
+    (1, 20, 20)
+
+The objects ``running`` and ``running_at_t1`` have different ids, but the same identity; they are different
+versions of the same object.  The id of the old version has changed; the new version has the original id value.
+
+Notice that ``stb`` and ``lca`` still refer to Discipline with id ``1``. When they were created, at t1, they were
+actually pointing to a different version than the current version.  Their discipline_id column was not updated to
+point to the old version when ``running`` was cloned.  This is an important implementation detail - foreign
+keys point to the latest version of the foreign object, which always has it's id equal to it's identity.  If this
+was not the case, it would be necessary to clone all of the objects that have a foreign key pointing to object X when
+object X is cloned; this would result in a very quickly growing database.
+
+When searching for an object at a given time t1, foreign key values are matched against the related records identity
+column, and the related record are further restricted to those records that are valid at t1.
+
+All of this should help you understand that when you filter a query for a certain point in time using an object,
+it's actually the identity of the object that will be used for the filtering, and not the id.  You are effectively
+saying, "I want to limit to records that were associated *with some version of* this object".
+::
+
+    >> stb1 = SportsClub.objects.as_of(t1).filter(discipline=running, name='STB').first()
+    >> stb2 = SportsClub.objects.as_of(t1).filter(discipline=running_at_t1, name='STB').first()
+    >> (stb1.discipline.id, stb2.discipline.id)
+    (2, 2)
+
+    >> stb3 = SportsClub.objects.current.filter(discipline=running, name='STB').first()
+    >> stb4 = SportsClub.objects.current.filter(discipline=running_at_t1, name='STB').first()
+    >> (stb3.discipline.id, stb4.discipline.id)
+    (1, 1)
+
+
+If you really want to filter using the id of the object, you need to explicitly use the id instead of
+passing the object itself::
+
+    >> stb5 = SportsClub.objects.as_of(t1).filter(discipline_id=running.id, name='STB').first()
+    >> stb6 = SportsClub.objects.as_of(t1).filter(discipline_id=running_at_t1.id, name='STB').first()
+    >> (stb5.discipline.id, stb6 is None)
+    (True, 2)
+
+    >> stb7 = SportsClub.objects.current.filter(discipline_id=running.id, name='STB').first()
+    >> stb8 = SportsClub.objects.current.filter(discipline_id=running_at_t1.id, name='STB').first()
+    >> (stb7.discipline.id, stb8 is None)
+    (1, True)
 
 Many-to-Many relationships
 ==========================
@@ -406,7 +484,7 @@ the current state, or the state at a specific point in time::
     local_members = club.members.filter(phone__startswith='555').all()
 
     # Working with a specific point in time:
-    november1 = datetime.datetime(2014, 11, 1).replace(tzinfo=pytz.utc)
+    november1 = datetime(2014, 11, 1).replace(tzinfo=utc)
     club = Club.objects.as_of(november1).get(name='Sweatshop')
     # The related objects that are retrieved were existing and related as of november1, too.
     local_members = club.members.filter(phone__startswith='555').all()
@@ -456,6 +534,42 @@ The syntax for soft-deleting is the same as the standard Django Model deletion s
     club.members.remove(person4.id)
     club.members = []
 
+.. _prefetch_related_notes:
+
+Notes about using prefetch_related
+----------------------------------
+`prefetch_related <https://docs.djangoproject.com/en/1.8/ref/models/querysets/#prefetch-related>`_ accepts
+simple sting lookups or `Prefetch <https://docs.djangoproject.com/en/1.8/ref/models/querysets/#django.db.models.Prefetch>`_
+objects.
+
+When using ``prefetch_related`` with CleanerVersion, be aware that when using a ``Prefetch`` object **that
+specifies a queryset**, that you need to explicitly specify the ``as_of`` value, or use ``current``.
+A ``Prefetch`` queryset will not be automatically time-restricted based on the base queryset.
+
+For example, assuming you want everything at the time ``end_of_last_month``, do this::
+
+    disciplines_prefetch = Prefetch(
+        'sportsclubs__discipline_set',
+        queryset=Discipline.objects.as_of(end_of_last_month).filter('name__startswith'='B'))
+    people_last_month = Person.objects.as_of(end_of_last_month).prefetch_related(disciplines_prefetch)
+
+On the other hand, the following ``Prefetch``, without an ``as_of`` in the queryset, would result in all
+matching ``Discipline`` objects being returned, regardless whether they existed at ``end_of_last_month``
+or not::
+
+    # Don't do this, the Prefetch queryset is missing an as_of():
+    disciplines_prefetch = Prefetch(
+        'sportsclubs__discipline_set',
+        queryset=Discipline.objects.filter('name__startswith'='B'))
+    people_last_month = Person.objects.as_of(end_of_last_month).prefetch_related(disciplines_prefetch)
+
+If a ``Prefetch`` without an explicit queryset is used, or a simple string lookup, the generated queryset will
+be appropriately time-restricted.  The following statements will propagate the base queries'
+``as_of`` value to the generated related-objects queryset::
+
+    people1 = Person.objects.as_of(end_of_last_month).prefetch_related(Prefetch('sportsclubs__discipline_set'))
+    people2 = Person.objects.as_of(end_of_last_month).prefetch_related('sportsclubs__discipline_set')
+
 Navigating between different versions of an object
 ==================================================
 
@@ -632,9 +746,9 @@ Postgresql specific
 ===================
 
 Django creates `extra indexes <https://docs.djangoproject.com/en/1.7/ref/databases/#indexes-for-varchar-and-text-columns>`_
-for CharFields that are used for like queries (e.g. WHERE foo like 'fish%'). Since Django 1.6 and 1.7 do not support
-native database UUID fields, the UUID fields that are used for the id and identity columns of Versionable models have these extra
-indexes created.  In fact, these fields will never be compared using the like operator.  Leaving these indexes would create a
+for CharFields that are used for like queries (e.g. WHERE foo like 'fish%'). Since Django 1.6 (the version CleanerVersion originally
+targeted) did not have native database UUID fields, the UUID fields that are used for the id and identity columns of Versionable models
+have these extra indexes created.  In fact, these fields will never be compared using the like operator.  Leaving these indexes would create a
 performance penalty for inserts and updates, especially for larger tables.  ``versions.util.postgresql`` has a function
 ``remove_uuid_id_like_indexes`` that can be used to remove these extra indexes.
 

setup.py

@@ -21,7 +21,7 @@
 """
 
 setup(name='CleanerVersion',
-      version='1.5.2',
+      version='1.5.4',
       description='A versioning solution for relational data models using the Django ORM',
       long_description='CleanerVersion is a solution that allows you to read and write multiple versions of an entry '
                        'to and from your relational database. It allows to keep track of modifications on an object '
@@ -37,7 +37,7 @@
       install_requires=['django'],
       package_data={'versions': ['static/js/*.js','templates/versions/*.html']},
       classifiers=[
-          'Development Status :: 4 - Beta',
+          'Development Status :: 5 - Production/Stable',
           'Framework :: Django',
           'Intended Audience :: Developers',
           'Programming Language :: Python :: 2.7',

tox.ini

@@ -5,13 +5,11 @@
 
 [tox]
 envlist =
-	py{27,34}-django{16,17,18}-{sqlite,pg}
+	py{27,34}-django{18}-{sqlite,pg}
 
 [testenv]
 deps =
 	coverage
-	django16: django>=1.6,<1.7
-	django17: django>=1.7,<1.8
 	django18: django>=1.8,<1.9
 	pg: psycopg2
 commands =

versions/admin.py

@@ -12,8 +12,7 @@
 from django.utils.encoding import force_text
 from django.utils.text import capfirst
 from django.template.response import TemplateResponse
-from django import VERSION
-
+from datetime import datetime
 
 class DateTimeFilterForm(forms.Form):
     def __init__(self, request, *args, **kwargs):
@@ -52,6 +51,7 @@ class DateTimeFilter(admin.FieldListFilter):
     title = 'DateTime filter'
 
     def __init__(self, field, request, params, model, model_admin, field_path):
+        self.field_path = field_path
         self.lookup_kwarg_as_ofdate = '%s_as_of_0' % field_path
         self.lookup_kwarg_as_oftime = '%s_as_of_1' % field_path
         super(DateTimeFilter, self).__init__(field, request, params, model, model_admin, field_path)
@@ -67,8 +67,9 @@ def get_form(self, request):
         return DateTimeFilterForm(request, data=self.used_parameters, field_name=self.field_path)
 
     def queryset(self, request, queryset):
-        if self.form.is_valid() and self.form.cleaned_data.values()[0] is not None:
-            filter_params = self.form.cleaned_data.values()[0]
+        fieldname = '%s_as_of' % self.field_path
+        if self.form.is_valid() and fieldname in self.form.cleaned_data:
+            filter_params = self.form.cleaned_data.get(fieldname, datetime.utcnow())
             return queryset.as_of(filter_params)
         else:
             return queryset
@@ -133,7 +134,7 @@ def get_readonly_fields(self, request, obj=None):
         This is required a subclass of VersionedAdmin has readonly_fields ours won't be undone
         """
         if obj:
-            return self.readonly_fields + ('id', 'identity', 'is_current',)
+            return list(self.readonly_fields) + ['id', 'identity', 'is_current']
         return self.readonly_fields
 
     def get_ordering(self, request):
@@ -164,21 +165,34 @@ def get_list_filter(self, request):
         Adds versionable custom filtering ability to changelist
         """
         list_filter = super(VersionedAdmin, self).get_list_filter(request)
-        return list_filter + (('version_start_date', DateTimeFilter), IsCurrentFilter)
+        return list(list_filter) + [('version_start_date', DateTimeFilter), IsCurrentFilter]
+
+    def restore(self,request, *args, **kwargs):
+        """
+        View for restoring object from change view
+        """
+        paths = request.path_info.split('/')
+        object_id_index = paths.index("restore") - 1
+        object_id = paths[object_id_index]
 
-    def restore(self, request, *args, **kwargs):
-        return True
+        obj = super(VersionedAdmin,self).get_object(request, object_id)
+        obj.restore()
+        admin_wordIndex = object_id_index - 3
+        path = "/%s" % ("/".join(paths[admin_wordIndex:object_id_index]))
+        return HttpResponseRedirect(path)
 
     def will_not_clone(self, request, *args, **kwargs):
         """
         Add save but not clone capability in the changeview
         """
         paths = request.path_info.split('/')
-
-        object_id = paths[3]
+        index_of_object_id = paths.index("will_not_clone")-1
+        object_id = paths[index_of_object_id]
         self.change_view(request, object_id)
+
+        admin_wordInUrl = index_of_object_id-3
         # This gets the adminsite for the app, and the model name and joins together with /
-        path = '/' + '/'.join(paths[1:3])
+        path = '/' + '/'.join(paths[admin_wordInUrl:index_of_object_id])
         return HttpResponseRedirect(path)
 
     @property
@@ -201,7 +215,8 @@ def get_object(self, request, object_id, from_field=None):
         """
         obj = super(VersionedAdmin, self).get_object(request, object_id)  # from_field breaks in 1.7.8
         # Only clone if update view as get_object() is also called for change, delete, and history views
-        if request.method == 'POST' and obj and obj.is_latest and 'will_not_clone' not in request.path and 'delete' not in request.path:
+        if request.method == 'POST' and obj and obj.is_latest and 'will_not_clone' not in request.path \
+                and 'delete' not in request.path and 'restore' not in request.path:
             obj = obj.clone()
 
         return obj
@@ -223,8 +238,7 @@ def history_view(self, request, object_id, extra_context=None):
             content_type=get_content_type_for_model(model)
         ).select_related().order_by('action_time')
 
-        ctx = self.admin_site.each_context() if VERSION < (1, 8) \
-            else self.admin_site.each_context(request)
+        ctx = self.admin_site.each_context(request)
 
         context = dict(ctx,
                        title=('Change history: %s') % force_text(obj),
@@ -246,7 +260,8 @@ def get_urls(self):
         Appends the custom will_not_clone url to the admin site
         """
         not_clone_url = [url(r'^(.+)/will_not_clone/$', admin.site.admin_view(self.will_not_clone))]
-        return not_clone_url + super(VersionedAdmin, self).get_urls()
+        restore_url = [url(r'^(.+)/restore/$', admin.site.admin_view(self.restore))]
+        return not_clone_url + restore_url + super(VersionedAdmin, self).get_urls()
 
     def is_current(self, obj):
         return obj.is_current