​http://s031.okserver.org:2375/user

needs some spacing between the image & user name

We could also display some stats here (e.g. number of dataset created, timestamp of last activity)

License titles at least for some of them need translating. This wants to be done in a more efficient and simple manner than currently exists.

the original problem was:

The license dropdown on the dataset form shows titles in Swedish by default. Changing the language seems to default to English, but after refreshing the page a couple of times it shows a wrong language again.

Thanks to pudo for reporting.

There is some evidence pointing to CKAN handling memory inefficiently or even leaking under certain conditions:

When we migrated ckan.net/thedatahub.org from eu7.okfn.org (32bit) to s053.okserver.org (64bit) (​ticket) we experienced extraordinary memory usage peaks (​ticket). Here are the observed value with Apache default settings:

• eu7, mpm-prefork: base level ~0.6GB, peaks up to 2GB
• s055, mpm-prefork: base level ~1GB, peaks up to 4GB
• s055, mpm-worker: base level ~1.5GB, peaks up to 6GB

William reduced the life-time of a WSGI CKAN process from 500 requests down to 25 requests (​changeset). This (together with two other tweaks) changed the situation drastically:

• s055, mpm-event: base level ~1.4GB, no peaks

This suggests that the more requests a CKAN processes serves over time, the more memory it consumes, aka bad memory management or a leak.

To prove this theory, one could reduce the total number of WSGI CKAN processes as much as possible without killing the performance (e.g. down to processes=3), and then observing the relation between maximum-requests=25...500 and memory consumption.

On 14/09/11 17:49, David Read wrote:

Someone to do a bit of top-down memory-use profiling would be very useful. Also useful would be something in the tests that reported what test cases use lots of memory - this could be in the nose plugin.

+1

TSV support doesn't seem to work very well, may be the mimetype ( text/tab-separated-values )

See ​http://thedatahub.org/dataset/wikipedia-e3-timestamp-position-modification/resource/d883ab44-07f4-4992-800a-3e4bf5d53a96

Instead of showing a notice, the password reset page throws an exception if the key parameter is missing:

Module ckan.controllers.user:329 in perform_reset
         c.reset_key = request.params.get('key')
               if not mailer.verify_reset_link(user_obj, c.reset_key):
                   h.flash_error(_('Invalid reset key. Please try again.'))
                   abort(403)
 if not mailer.verify_reset_link(user_obj, c.reset_key):
Module ckan.lib.mailer:100 in verify_reset_link
     if not user.reset_key or len(user.reset_key) < 5:
               return False
           return key.strip() == user.reset_key
 return key.strip() == user.reset_key
AttributeError: 'NoneType' object has no attribute 'strip'

Apart from the obvious fix of checking for the 'key' parameter, it seems like is quite common to get these reset urls without the key parameter, so I suspect some email clients might strip the query params when building the links. We could avoid this problem by making the key part of the url instead of a param:

​http://thedatahub.org/en/user/reset/3086e91c-fe09-4a98-92e1-19de67a9ac9d/b4c2d03fa8

instead of:

​http://thedatahub.org/en/user/reset/3086e91c-fe09-4a98-92e1-19de67a9ac9d?key=b4c2d03fa8

in create/edit dataset forms

When adding a resource, the breadcrumb should say "add data" or "add resource" not "create dataset" as you see here: ​http://s031.okserver.org:2375/dataset/new_resource/frere

Should be basic format (which is what the datastore wants & what we display to users) e.g. "CSV, XML, JSON etc"

​http://s031.okserver.org:2375/dataset/new_resource/

templates/error_document_template.html

I've converted the 404 page to the new style but we probably want to serve this as a 'full page' ie content is 100% page width not sidebar and primary content

just go to a non-existant url to see

Discusion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/63374042/

Implementation:

​http://s031.okserver.org:2375/404

From DR at DGU

Basically get_groups() appears to cache its results, but this falls down when subsequent calls are in a different session. We get this when saving a dataset. get_groups() first gets called in the controller when it is trying to work out what permissions the user has. It later gets called, and by this time c.userobj is detached, so get_groups() fails.

I'm working around this by refreshing c.userobj (from c.user) before calling get_groups(), but I wonder if the get_groups caching could detect the detached session and bypass the cache in this case?

diff at ​https://gist.github.com/3003117

I'm not seeing any javascript or python error, the upload just hangs.

IDatasetForm's form_to_db_schema() method is not getting called by ckan

In branch feature-1821-multilingual-extension, a new faceting datastructure was introduced. This makes the old one and related functions obsolete. These have been marked as deprecated for the 1.7 release, and should be removed for the 1.8 release.

• helpers.py:facet_items()
• facets.html:facet_sidebar()
• uses of c.facets (rather than c.new_facets)

Need a central up to date place for currently supported extensions (closest we have is ​http://wiki.ckan.org/List_of_Extensions) & their documentation. I suggest this gets added to a dedicated section of docs.ckan.org

see ​http://s031.okserver.org:2375/en/dataset/test-dataset/resource_edit/9d27a9d9-36ec-460e-9edb-6dff7ba4fc28

& image attached

Requirements

Would be useful if we could extend the Google Analytics extension to push track events to GA so that we can determine how the system is being used, and where we can optimise the workflow.

Assuming it isn't configured to be off, track events should be added to some key action buttons so that the information on what users are doing is logged for analysis.

Interface

This could possibly just be a list of JS calls to $('x').click(_gaq.push('track'...))

User Stories

• As a developer I would like to know which components of the system are used the most so I can streamline the processes.

• As a support developer I would like to know which parts add complexity to the system for minimal gain and could possibly be removed, de-prioritised.

For which metrics, as many of the following as possible that are achievable without changing the templates. I think for now it is enough to just record counts of these pages/actions.

• Any group link on homepage

• Clicking on username (to go to profile)

• In user profile, clicking on Edit Profile

• Clicking Save Changes on Edit Profile page

• Clicking on any dataset link on User Profile page

• Any of the group links on /group

• Clicking any of the right hand sidebar tags on /group/X

• Visiting /group/history/X

• Compare Button on /group/history/X

• Compare Button on /dataset/history/X

• Tags on right hand sidebar of /dataset/X

• Download button on any /dataset/X/resource/* page

• Data API button on any /dataset/X/resource/* page

Tasks

[ ] Analysis/Clarification?

[ ] Code/Config? changes

[ ] Documentation

Estimates

Pricing <ckan.org/datasuite/solutions> needs to go to <ckan.org/datasuite/services>

http://ckan.org/datasuite/services/

My feeling is that 'country-US' and 'country-us' should be the same tag. However currently tags with caps are treated differently

see ​http://s031.okserver.org:2375/en/dataset/test-dataset

with TEST and test - there also get indexed twice in the search page

Results in bad user experience and WebApp? errors emailed out. Seen in 1.7.1

hide trackbacks & subscribe wboxes on dataset pages until we add disqus

in 2375-demo-theme-development

templates/package/confirm_delete.html needs to be made to look nice note some writing in black

delete button needs nice home on dataset/edit

you can add js support add ?confirm=yes to url on dataset button to force delete

Please note delete is not yet implemented - but will be in the next few hours

For example, hide the 'edit' button on user profiles ​http://s031.okserver.org:2375/user/ross2#activity unless I'm logged in as Ross

we need a delete_resource logic action and enable for resources

Needs to be a way for admins (owners of dataset) to delete related items. I suggest adding a button to the edit page

​http://s031.okserver.org:2375/dataset/gold-prices/related/edit/d3249863-040a-4f55-a5d8-c86697d0b7b8

contextpackage_limits? = { 'tags': 5, <- get first 5

'extras': 0, <- get all

}

only get what you ask for have to be explicit

contextgroup_limits? = {} only main item

start with datasets/groups expand if we like it

Something to make building breadcrumbs a bit nicer

the "add your own data" link in "Try another search term, browse the datasets below or add your own data." here ​http://s031.okserver.org:2375/dataset?q=stuff doesn't take you to add a dataset, just back to the search/datasets page

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61696765/comments

404 Not Found The resource could not be found.

I think it would be nice to have all form pages monitor their forms - excluding search ones - and if a field has changed then warn the user that they have not saved the form data but are leaving the page.

This would help with things like changing resources on the add dataset resource form etc.

Currently the group_index page just shows the entire list of groups, forcing the ordering to be by name. It would be better if it could be sortable by name (or reversed) or by package_count (or reversed)

Clicking on the blue buttons or images on related page ​http://s031.okserver.org:2375/dataset/gold-prices/related should take you to a new tab instead of navigating you away

Stuff I've spotted:

• forms/*
• template/group/edit_form.html
• template/package/edit_form.html

This can go once new DGU form is in.

Far too many resource options. Lets restrict back to data file and API. Visualizations etc can either get linked in description or in the Related items.

• Only logged in users able to create
• Creation permissions have only creators able to update a dataset
• Synchronous search is enabled (#1678)

david I've got the add resource working but it is happy to add empty resources which implies that it isn't doing validation correctly.

I'm not sure what the validation rules are so hard to test this - I'm coming through as an api call

let me know if i can help

same as yesterday: ​http://demo.ckan.org/dataset/edit/newcastle-city-council-payments-over-500 clicking update/save returns error

See ​http://demo.ckan.org/dataset/example-dataset/resource/d8797e51-b497-46ca-a274-8675533d110b

David previously fixed this on some version of demo, seems to have regressed. Just want the window to be longer

the stages snippet needs to be inside <form> so I can make some links buttons - eg stage 1 -> stage 2 I want to save the dataset not just move to add resource

if you can just move it within the form html leaving everything else the same - I can then add buttons and pass back for styling

For example.

1. Start on stage one
2. Add dataset click next
3. Click previous

Expect Stage 1 to be "active", stage two to be "complete"

1. Start on stage one
2. Add dataset click next
3. Click next
4. Click previous

Expect Stage 2 to be "active", stage one and three to be "complete"

1. Start on stage one
2. Add dataset click next
3. Click next
4. Click previous
5. Click previous

Expect Stage 1 to be "active", stage two and three to be "complete"

make sure documented

Allow user to navigate directly from one resource page to another via resource box in sidebar

• Show resources in the sidebar
• Allow people to go backward and forward
• Have a back button on each form
• Lose resource type
• Add extras as optional metadata
• Remove save
• Add back/Previous
• Add Another

See the related sidebar box

​http://s031.okserver.org:2375/dataset/adur_district_spending

If a package has related items the first (or a random) related item should be passed into the snippet.

changing a tag from "country-uk" to "country:uk" and updating gives server error

otherwise updating metadata / adding tags seem fine

either we should allow colons or form should return a message indicating what characters are allowed

make it so

In the footer: Change 'Powered by CityData?' to Powered by DataSuite? (with link to ckan.org/datasuite)

Links in left hand side of footer should change to a link to ckan.org, link to OKFN, link to ckan.org/datasuite/about and link to Pricing Page (on ckan.org .. will be /datasuite/solutions)

In create dataset form, prefilled tag examples should include one example with a space, e.g. "mental health"

Change 'Groups' to 'Publishers' everywhere

Homepage copy/images (Ira to supply?)