CKAN: Ticket #1129: CREP0002: Moderated Edits

attachment set

kindly — Sun, 08 May 2011 10:49:02 GMT

attachment set to sqlspeed.py

script run do test speed of various ways of doing a "last before this date" query

priority, state changed; cc, milestone set

thejimmyg — Wed, 11 May 2011 12:20:19 GMT

cc rufus.pollock@… added
priority changed from awaiting triage to major
state changed from draft to accepted
milestone set to ckan-v1.5

Great proposal, here's my thinking:

The current VDM model (where we have things like a single package table which logs changes to a package_revision table) is very well tested and works well and we should only consider a radical change with good reason
Although we are making a move from *model objects* being the central components to the *logic layer functions* being the central components, there is no need yet to have the database structures we store (for revisioning or otherwise) start to look like the dictized representation. The logic layer itself is designed to do that mapping.
Since we do have this logic layer though, there is less need for "magic" SQLAlchemy objects to support dicts and lists (eg the stateful objects used to make a list of tags on a package behave like a list), a few simple (and easily debuggable) queries in the logic layer would work better.

In the longer term we may want to look at serialising more dictized-looking objects and that may lend itself to a more dictized changeset model or even a more dictized storage system (eg no-SQL) but for the time being we are not at that point.

I recommend the following:

We continue to use the current default branch of VDM (not the changeset branch)
We continue to treat the package table (and other non-revision tables) as the most recent revision (even though the *active* revision displayed in CKAN might well be in the revision tables because more recent changes haven't been moderated yet)
We slowly stop using stateful lists and dicts in CKAN because we have move control with explicit queries in the logic layer

Sound good?

rgrp — Wed, 11 May 2011 18:53:00 GMT

It seems like this ticket is getting a bit confused with ticket:1076 (changes re vdm) :-)

Some general conclusions relevant to both:

General feeling is not to change to changeset model
- I'm personally +0 on changeset model
- I would note main benefit is simplicity and orthogonality of changesets to core system. I also think cost of change is small.
- That said we can simplify current model as well.
There seems some misunderstanding: change to have logic layer has almost nothing to do with being able to remove main stateful stuff in vdm. To be able to remove most of stateful stuff in vdm requires us to make some other changes (re foreign keys from revision objects to continuity)
There are other simplifications we should make to vdm before embarking on this (e.g. move to SessionExtension? from MapperExtension?). This is easy as that work has been done in changeset branch and can be backported.

What I don't see in this CEP as yet is any discussion of the complexities around pending stuff (e.g. do we allow multiple pending, do new edits get shown current or latest pending). I'll try and comment more on this but a lot of this was in original vdm discussion last summer.

rgrp — Thu, 12 May 2011 14:13:21 GMT

Replying to rgrp:

It seems like this ticket is getting a bit confused with ticket:1076 (changes re vdm) :-)

I of course ;0 meant ticket:1077

kindly — Thu, 12 May 2011 15:01:07 GMT

There seems some misunderstanding: change to have logic layer has almost nothing to do with being able to remove main stateful stuff in vdm. To be able to remove most of stateful stuff in vdm requires us to make some other changes (re foreign keys from revision objects to continuity)

The logic layer does not automatically help out, however it makes our life easier if we want to handle state ourselves. For example take package tags, if we remove the stateful_m2m properties and just use normal sqlalchemy relations. We will still want statefullness (i.e active, pending, deleted) on the package_tags table. We should update those on the table ourselves in the logic layer.

There are other simplifications we should make to vdm before embarking on this (e.g. move to SessionExtension? from MapperExtension?). This is easy as that work has been done in changeset branch and can be backported.

I agree but event thought the MapperExtension? way is not great, it is very well field tested.

description changed

kindly — Thu, 19 May 2011 19:07:48 GMT

description modified (diff)

attachment set

kindly — Thu, 19 May 2011 19:18:38 GMT

attachment set to sudgested_vdm.svg

attachment set

kindly — Thu, 19 May 2011 19:19:16 GMT

attachment set to suggested_vdm.png

status changed; resolution set

thejimmyg — Fri, 08 Jul 2011 12:41:02 GMT

status changed from new to closed
resolution set to fixed

rgrp — Fri, 30 Dec 2011 18:01:40 GMT

More UI work remains in #1141