In branch feature-1821-multilingual-extension, a new faceting datastructure was introduced. This makes the old one and related functions obsolete. These have been marked as deprecated for the 1.7 release, and should be removed for the 1.8 release.

• helpers.py:facet_items()
• facets.html:facet_sidebar()
• uses of c.facets (rather than c.new_facets)

Change login cookie from a default expiry of 50 years to 2 years. You can also uncheck a 'remember me' checkbox on the login form for the cookie to just last the session.

Background conversation on ckan-dev:

DR: I wonder if anyone objects to the expiry of the login cookie to be changed from 50 years to 2 years? 50 years might be appropriate for thedatahub.org, but for government sites it seems (to me) to be too lax.

Toby: is this the repoze.who cookie? If so that seems sensible to me.

Rufus: Definitely agree. I would also like to see introduction of a standard "remember me" checkbox (set to true by default). At the moment a login lasts forever (until you logout) automatically.

• Add home page url
• Remove group option (seen it confuse several users) - perhaps replace with tags (?)
• Check form looks ok in portugese (seems to be a problem with css)

Use jquery chosen?

Apparently auto-complete in Organizations may not work. Check and fix.

See ​https://github.com/okfn/ckan/commit/5eca7d5e37c0ef392b768b8b3768b2c3f93448b5

• Add Ascii folding filter to text fields.
• Add capacity field for public, private access.
• Add title_string so you can sort alphabetically on title.
• Fields related to analytics, access and view counts.
• Add data_dict field for the whole package_dict.

Actual implementation of data_dict adding will be added later.

A place to list crucial fixes for v1.7 release:

Related extension:

All not complete now moved to #2347

• Related Item has comments on index page (they should not show up here -- why is this happening!)
• Related item should be to left of history (history could in fact move to RHS)
  • What about losing icons since we duplicate for related item
• Related item plus sign is very poorly aligned within box
  • Add related items should be in drop down like for resources (?) to make it easy to add items
  • Hover should show a title that gives some information about what related are (not at all clear)
• Related item only includes App, Idea, Visualization but not API | Post | Paper | News Article options as in original ticket #2204

Search results:

• #2331 - search should add not or
• Search results should be similar across group pages and normal search results (suggest adopting group page style)
  • Search results resource link should link to resource page rather than url for that resource
  • Suggest we could tidy this a bit (e.g. have resources in mini-sidebar of search result ...?)

Data viewer:

• Merge updates to Recline (that did go in before deadline but do not seem to have made v1.7 branch) ​https://github.com/okfn/ckan/commit/146a5c1130cc77cb2ad2c6ef6b3a53074908bf07
  • Ensure still using read-only (this seems to have been lost in upgrade)
• Do not show failed preview with red-bar (esp when just empty datastore ...)

a) Data Viewer checks if datastore is empty nad falls back to ther options if emtpy b) DataStore? enabled is off by default and only turned on either explicitly by user or by DataStorer? on successful store ...

• (? not really a fix) Give out a link as well as embed instructions (I often want to link to the explorer in a given state)
  • Should this link to the resource page or to a dedicated viewer page - cf # 2322 (resource sub-urls like /download, /api ...)
  • Do #2322 ...
• Decent powered by icon for the Embed for the DataHub? (make this text customizable and allow html ...)
• Embed link uses uuid rather than dataset name (this goes to perma-url question ...)
• Data viewer embed url includes all the resource info

Dataset page:

• Put home page link somewhere more prominent (e.g. the top of the sidebar)
• Put author more prominent (at top right)
• (? - new work) List CKAN owners at top right in sidebar
• Make resources stand out more
• Rename resources tab and heading to Data & Resources

Dataset edit and create:

• Corrections to dataset creation form for v1.7 - #2317
• Better auto-complete for groups on dataset edit/new page - #2319
  • Remove groups box and replace with tags on the new page (or is this needed for organizations stuff?)
• Description / notes text area is not wide enough.
  • Should also be larger on edit page
• Problem with date instructions on add/edit resource: "Dates are in ISO Format — eg. 2012-12-25 or %c2010-05-31T14:30%d."
• Add/edit resource: new code to auto-check urls being added is great but from UX point of view has draw back that (esp on low speed connections) page just seems to hang for a long time with nothing happening. I wonder if either
  • A) this could happen after resource created on form (e.g. it creates, and then an overlay comes up saying: checking url and gathering url information or similar)
  • Or B) we still do this before creating new resource form on page but we still should show something like a spinner and some info like "checking url and gathering url info ...)

Resource view:

• Do not show no description in no description (just show dataset info)

Theme:

• Fix spacing above page title (i.e. dataset title etc)
• Bootstrap 2.0.2 (@zephod was *supposed* to have done this) - should be trivial and gives important bugfixes for data viewer
• Fix subnav width on resources pages (does not extend all the way to the right like it should)
• Fix spacing of top menu and search box (do not look right)
• Group pages:
  • view page: image / logo is really poorly locate and sized, see e.g. ​http://datahub.io/group/wikimedia
  • view page: admin heading on RHS sidebar is wrongly indented

Miscellaneous:

• Ensure that we out-of-the-box config for CKAN has (#2388):
  • Only logged in users able to create
  • Creation permissions have only creators able to update a dataset

Deployments (without deployment we cannot know these are working):

• Social extension
• Analytics

• convert non-free fields to vocab fields (eg: interoperability_level)
• store values as JSON files in the ecportal repo
• translate output on dataset view page
• add tests

Some additions to the related stuff functionality to make it more useful post the 1.7 release.

[x] Edit item (after checking auth)

[x] My related things on my user page

[x] Show user created related things on the users' page with links direct to the related page of that dataset. {0.5d}

[x] Related stuff usage counts { 1d }

Record the number of times the link for a related item is clicked. This is likely to mean that the link fires to call CKAN and then redirects to the actual URL.

[x] Un-attached related items {0.25d}

Allow related items to be created without being attached to datasets. This may mean that the dataset field is optional in the API.

[ ] Dashboard for related

A dashboard for related items, site-wide (not package specific) and orderable by date, views, {rating}, popularity of the dataset. Would be useful (possibly) for site-admin to mark one or more as 'featured'

[x] Dashboard + Controller (Related) {1d}

[x] Ordering at logic layer (date, views, rating, popularity of package, featured) {1d}

[x] date [x] views [x] featured

[x] 'Featured' related item with mechanism to fetch it. {0.5d}

[ ] UI within dashboard for sysadmins to check/uncheck feature status {0.5d}

[x] Changing auth for related.

Currently auth is based on deletion being possible by dataset owner and the related owner but it may be that a lot of the auth for related items should be configured to have more restrictive auth.

[x] Implement auth changes for publisher auth profile. { 0.5d }

[x] General issues

[x] Serverside validation of URLs

[x] Activity updates

[x] Make optional

[x] Related item only includes App, Idea, Visualization but not API | Post | Paper | News Article options as in original ticket #2204

[ ] UI Changes

[ ] General tidy up, fix form etc (icon for menu bar)

[ ] Highlight featured items py:if="related.featured"

[ ] Show either view count or popularity ${ related.view_count }

[ ] A user's related items should be on their home page

[ ] Related item plus sign is very poorly aligned within box

[ ] Add related items should be in drop down like for resources (?) to make it easy to add items

[ ] Hover should show a title that gives some information about what related are (not at all clear)

"Apps, Ideas, Visualization and other material related to this dataset"

[x] Related is a pretty terrible name in the UI - much better to have it called Apps, Ideas etc (perhaps title tag could even give more details e.g. "Apps, Ideas, Visualization and other material related to this dataset") -- btw i thought this was in #2332 but realize it wasn't (apologies for that!)

• Went with Apps & Ideas (for now)

[x] Documentation, documentation, documentation - AFAICT I can't see anything in v1.7 or master docs. I imagine this would be a short section like ​http://docs.ckan.org/en/ckan-1.7/commenting.html (and probably coming right after that in the ToC) which says

• What it does
• How to enable (and perhaps includes a nice-screenshot!)
• How to customize (e.g. can one customize the list of options of things one can create (e.g. can I set it to just be app and idea or ...)
• Also liase with Mark W re a blog post about this from the user perspective (a screenshot walkthrough ...)

[ ] Clearly mark the old extension as deprecated from v1.7 forward: ​https://github.com/okfn/ckanext-apps (I just met someone last week who was working on integrating this and had no idea it was replaced by something better in v1.7)

Install and document (step-by-step) CKAN (and dependencies, and extensions) onto VM on s031.okserver.org which has CentOS 6.2 installed. Paying particular attention to the fact that we won't have root access on the staging nor production servers, and so a) anything that requires root access must be documented carefully; and b) we want to be able to perform maintenance without being root.

Maintenance

We want to have non-root access to:

• CKAN logs; (relevant) apache logs; qa process' logs; rdf cron logs
• restarting CKAN
• restarting qa processes
• changing/disabling rdf cronjob
• our database (through sql)
• restarting solr

CKAN and Dependencies

• CKAN installed from source
• Required plugins: synchronous_search ecportal ecportal_form organizations multilingual_dataset multilingual_group multilingual_tag qa

Extra configuration

• uploads
• analytics (toby)
• qa
• rdf cron
• caching
• datastore

Things to remember

• Use ckanext/multilingual/solr/schema.xml and stopword files
• Use nginx as proxy.

Provide some provisional user stories for the CMS/Blogging feature. Should include some of the functionality from wordpresser but be more generic.

• Do an analysis of why upgrades are problematic
• What can we do better
• Decide what to implement

Various stuff related with release 1.7:

• Review patches sent to the release branch
• Regularly update TDH with latest changes
• write changelog, announce email, etc
• ...

I've been writing a doc with different issues regarding INSPIRE support in CKAN. It is mostly done, except for some sections on CSW support. Depends on #2359

Need to investigate alternative payment gateways

1.7 will hide the link on the footer and put a deprecation notice on the related pages, but they should be completely removed on 1.8.

Solr install instructions install the wrong version

aims

• allow templates to specify needed css/js

• allow split js files to be bundled (including plugin supplied if possible - stage 2)

• serve minified in production/ non-minified in dev

• possibly look at doing the minification via paster - maybe as different ticket

e.g. 'Cabinet Office' returns 0 results, whereas 'Cabinet' returns 'Cabinet Office'.

/api/2/util/authorizationgroup/autocomplete?q=Cabinet%20Office

This feature only seems to be used by DGU at the moment (I think)

As designed by Sam Smith -> ​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/ Also to be used as the demo theme although I think they will diverge at some point. At the moment they should be implemented as one and the same.

Provide a new test install of DataGM based on the ODS demo site being completed through June.

Currently you can specify the capacity in the API search, this should be stripped and default to public for now.

US-0a

Do not show failed preview with red-bar (esp when just empty datastore ...)

• Data Viewer checks if datastore is empty nad falls back to ther options if emtpy
• DataStore? enabled is off by default and only turned on either explicitly by user or by DataStorer? on successful store ...

Loading/showing error reports

Map licenses: given ​http://creativecommons.org/licenses/by/2.5/{countrycode}/ as a license value through the API, convert it to ​http://www.opendefinition.org/licenses/cc-by . Should also work if given ​http://www.opendefinition.org/licenses/cc-by . NOTE: the API may send through a list of licenses; in this situation, take the head of the list.

One user reports that after resetting the password via the reset form he was unable to log in again. He did have an email address associated and received the reset email. After updating his password, login failed

"Login failed. Bad username or password. (Or if using OpenID, it hasn't been associated with a user account.)"

Search results are slow as we do many package_dictize. Store the package_dict in the search index so that we do not have to hit the database for each of them.

The settings > extras page for a dataset allows extra fields to be added, but only four can be added a time in a single atomic update. It would be helpful to have an 'add another' button on the form that can be used any number of times.

Taken from ​http://lists.okfn.org/pipermail/ckan-dev/2012-May/002218.html

A few months ago, the RDF provided when dereferencing a dataset's URI was very useful. It described the datasets using many of the "right" RDF vocabularies (dcterms, void, etc).

The RDF that is returned now no longer has these nice descriptions. Instead, it seems to be a mindless transcription of the CKAN record.

For example, the older descriptions gave:

<​http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>

void:subset [

void:target <​http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>, <​http://thedatahub.org/dataset/geonames-semantic-web> ; void:triples 50 ; a void:Linkset

], [

void:target <​http://thedatahub.org/dataset/dbpedia>, <​http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> ; void:triples 52 ; a void:Linkset

], [

void:target <​http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>, <​http://thedatahub.org/dataset/govtrack> ; void:triples 52 ; a void:Linkset

] ; void:triples 130005 ;

but now it's simply:

<​http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>

dct:relation [

rdf:value "130005" ; rdfs:label "triples"

], [

rdf:value "52" ; rdfs:label "links:dbpedia"

], [

rdf:value "50" ; rdfs:label "links:geonames-semantic-web"

], [

rdf:value "52" ; rdfs:label "links:govtrack"

] ;

My old copies of the dataset descriptions mentioned a "GoLD CKAN 1.0" that was credited for the nicer modeling.

Where did that go? Can we get it back?

Thanks, Tim Lebo

<​http://semantic.ckan.net/record/91d2c0de-75a4-4bb6-b260-bc2946e1be8b>

dc:source <​http://ckan.net/> ; opmv:wasGeneratedBy [

opmv:used <​http://ckan.net/api/rest/package/farmers-markets-geographic-data-united-states>, <​http://ckan.net/api/rest/revision/9c2545b8-ebf1-42bf-ac92-633ff5ceba3f> ; opmv:wasControlledBy [

a foaf:Agent ; foaf:name "GoLD CKAN 1.0"

] ; opmv:wasPerformedAt [

a time:Instant ; time:inXSDDateTime "2012-01-31T05:34:32Z"^^xsd:dateTime

] ; a opmv:Process

] ; a dcat:CatalogRecord ; dcat:dataset <​http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> ;

…

Core CKAN currently has a dependency on LXML and it would be nice if we could remove it (although this *may* be indirectly through genshi).

Currently facets in the new demo site disappear from the sidebar once they are selected.

​http://s031.okserver.org:2375/dataset?tags=fibre&tags=terrestrial&q=Africa

Instead they should be marked with an "active" class and clicking the link should remove the filter in the same way as clicking the cross in a token below the search field works.

create a depreciated decorator to wrap functions we intend to remove

logs a warning maybe check that 'depreciated' is included in the docstring too

Reported in IRC by @floapps

In previous ckan versions i could use paster commands to remove reader rights from visitors on all packages and this would then throw the user to login screen when trying to see a dataset but, when i did this in ckan 1.7, i could still see all datasets as a visitor

Using ​http://docs.ckan.org/en/latest/authorization.html#permissions-publisher-mode

paster rights remove visitor reader package:all

Also has an empty list of roles in config for default visitor roles

I tested on my local install and could replicate with that command doing nothing. Neither affects existing or newly created datasets.

This ticket depends on #2423, #2424, #2425, #2426 and #2427.

• Try to remove as many of CKAN's dependencies as possible

• Try to update as many of the remaining ones as possible

• Try to get rid of some dependencies that are fetched from random hg or git servers instead of pypi

• Then try to get the order of deps in the requirements files right so that you end up with the right versions of everything

• Each module in the dependencies file should specify the exact version, to protect us from changes due to them upgrading versions under us and breaking things or changing their own deps

• Consider shipping dependencies with ckan instead of pip installing them?

• This is related to moving CKAN to Ubuntu 12.04

• Add a test case that runs pip freeze and tests that all the right versions of deps are installed?

We want a new way of working with trac tickets and milestones etc:

• We will have a single queue of tickets, ordered by priority, that we all work on

• Tickets only get into the queue after triaging (which the devs do at regular intervals), so we and others can use new trac tickets as a dumping ground for any and all CKAN bugs and ideas without messing up the priority queue. Therefore the priority queue needs to be a special CKAN milestone or something. The devs need to regularly go through the inbox of untriaged tickets and triage them all, moving them all out of the inbox.

• Will we also assign tickets from the priority queue to individual devs in sprint planning meetings like we currently do?

• [ ] Write up python coding standards as discussed at dev-meetup into CKAN docs. (Mostly written-up as part of dev-meetup notes.)

• [ ] Collate frontend coding standards (​http://aron.github.com/ckan-style/styleguide/) into same doc(s).

tag_search and tag_autocomplete: The fields argument is not available when accessing this action with a GET request.

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61906351/comments

Implementation:

​http://s031.okserver.org:2375/dataset/

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62308209/comments

Implementation:

(to be added once test server is working)

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62821386/comments

Implementation:

​http://s031.okserver.org:2375/dataset/adur_district_spending/related/new

Needs a new endpoint at /dataset/DATASET/related/new this should have a form containing the following fields.

• Title (required)
• Type of item (Application|Visualisation)
• Description
• URL (required)
• Image URL

When submitted if an item is created it should redirect (303) back to /dataset/DATASET/related with a flash message saying "Related item was successfully created".

If failed to create it should redirect (303) back to the form and populate the error messages, see add dataset for examples (​https://github.com/okfn/ckan/blob/feature-2375-demo-theme/ckan/templates/package/new_package_form.html)

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61911712/comments

Implementation:

​http://s031.okserver.org:2375/group