with add button on dataset page and in 'data & resource' section when there's no data

i.e. mimetype, size, last updated etc

​http://s031.okserver.org:2375/dataset/hi-people/resource/4558c365-fd7a-4f0e-9f2a-90ad47db6b39

If you modify your user name, it looks like you are logged out, but you are actually logged in and therefore can't login. (Reported by Pablo)

As a work around, you need to logout and log back in.

implementation

• We should force this by auto-logging you out (DR: maybe!)

which is confusing since you can only search for one tag like this at a time. I.e. tags:economics,cvs or tags:economics, csv or tags:economics+CSV doesn't work for example; therefore tag:economics, should also work!

​http://s031.okserver.org:2375/dataset?q=tags%3Aeconomics&sort=relevance+asc

1. If a user enters the wrong format on a file that can be previewed - it simply won't be previewed (e.g. a CSV or XML file that can be filled in with JSON in format will just not work or check this

2. If I incorrectly edit format to one that data preview will try to preview it will work even for a format that it doesn't accept (sometimes) e.g. this PDF file I changed the metadata to HTML ​http://s031.okserver.org:2375/en/dataset/test-dataset/resource/9d27a9d9-36ec-460e-9edb-6dff7ba4fc28

If a user enters a url with no protocol (eg: www.data.gov.uk) in the 'url' field of a dataset, a broken link is created on the view page.

CKAN should probably either put in a ​http:// if no protocol is given, or validation should fail.

A large number of XXX.ckan.net sites give the following message:

"This Site is Down for Maintenance We apologize for the inconvenience. ~ The Open Knowledge Foundation sysadmins."

The message is unhelpful and patently false - the sites do not exist. Some of them were supposed to have been redirected to a relevant group at thedatahub.org in this ticket (now closed):

​http://trac.okfn.org/ticket/933

However, the redirection only seems to have worked in one case, ​http://si.ckan.net.

The problem still affects the following sites - the first 4 of which have supposedly been merged:

• ​http://lt.ckan.net
• ​http://hu.ckan.net
• ​http://be.ckan.net
• ​http://gr.ckan.net

• ​http://nz.ckan.net
• ​http://bg.ckan.net
• ​http://fi.ckan.net
• ​http://katalogdanych.centrumcyfrowe.pl
• ​http://pl.ckan.net
• ​http://southampton.ckan.net
• ​http://sl.ckan.net

Please sort this out by redirecting, removing the sites, giving a more helpful (and accurate) failure message, etc, as appropriate.

I thought I'd made this ticket before but can't find it. When we have a fully created dataset we need to be able to edit the resources.

template: package/resource_edit.html

when you've made a form reassign to me and I'll get it working

According to sqlalchemy docs [1], when calling Session.is_modified(), the passive argument needs to be passed in as True.

[1] ​http://docs.sqlalchemy.org/en/rel_0_7/orm/session.html?highlight=is_modified#sqlalchemy.orm.session.Session.is_modified

• [ ] Write up python coding standards as discussed at dev-meetup into CKAN docs. (Mostly written-up as part of dev-meetup notes.)

• [ ] Collate frontend coding standards (​http://aron.github.com/ckan-style/styleguide/) into same doc(s).

Why is datastore not enabled for this resource ​http://s031.okserver.org:2375/dataset/frere/resource/cdeddb39-b0cb-4762-aa5b-f80fc892d4dd ?

Is this going to be a site wide setting? How do new resources get added to datastore? Or are we just using the dataproxy for demo?

Steps to reproduce:

1. go to edit a resource, e.g. ​http://s031.okserver.org:2375/dataset/ted/resource_edit/a621a6ad-6f1c-46d1-b858-03cdb7a25c07

2. Change the format field to something it doesn't get. e.g. add a comma "HTML,"

3. Get message: "The form contains invalid entries:

Last modified: Date format incorrect Size: Invalid integer"

4. Format field reverts, try to re-save/submit form - i.e. click on "update resource"

5. nothing happens (same if you even try changing the format field

Currently the 'add dataset' button on a group page ​http://s031.okserver.org:2375/group/another-group-in-the-wall goes to edit group. This is confusing since the interface for this is no way streamlined or helpful.

Can we get this to go to the create dataset process instead (ideally with the group pre-chosen) for now?

Further on will need to re-think & plan the workflow for publishers which will be a large part of new work

This is pretty key for standardization.

Also would like us to change the grey text to be: e.g. CSV, JSON, [more examples... i have no idea what they key ones are tbh]

Text goes over the order by text, with drop down floating on top meaning it's all impossible to read

​http://s031.okserver.org:2375/dataset?q=unemployment%2C+data&sort=metadata_modified+desc

in dataset creation / edit form

You can reset your password (#1186) but you have to know the URI (/user/reset) - there is no link to this page!

​https://github.com/okfn/ckan/blob/master/LICENSE.txt mentions requires/lucid_conflict.txt which is no longer the case for 1.8 and needs updating

not sure where requirements-testing/docs fit in

@sean I've assigned this to you as you changed the requirements and my plate is full feel free to pass on to someone else

Hey, this is still missing. It's the one with the add dataset text

For example here: ​http://s031.okserver.org:2375/en/dataset/new_resource/gold-prices The breadcrumb shows "gold-prices" which is not the title of the dataset

​http://s031.okserver.org:2375/dataset/new

license dropdown - when open we need some line spacing at least in FF I think it'd be nice to fix this in phase 2

Currently for some reason I'm not getting the errors/data dict in the package/new.html template.

This should cover formatting and best practices for HTML, CSS and JavaScript?

Perhaps this should sit in it's own repository (ckan-styleguides?) or gh-pages branch.

When navigating to ​http://s031.okserver.org:2375/group on the demo server, the Group link in the navbar is not highlighted in the same was as the other links are when visiting those pages.

Some additions to the related stuff functionality to make it more useful post the 1.7 release.

[x] Edit item (after checking auth)

[x] My related things on my user page

[x] Show user created related things on the users' page with links direct to the related page of that dataset. {0.5d}

[x] Related stuff usage counts { 1d }

Record the number of times the link for a related item is clicked. This is likely to mean that the link fires to call CKAN and then redirects to the actual URL.

[x] Un-attached related items {0.25d}

Allow related items to be created without being attached to datasets. This may mean that the dataset field is optional in the API.

[ ] Dashboard for related

A dashboard for related items, site-wide (not package specific) and orderable by date, views, {rating}, popularity of the dataset. Would be useful (possibly) for site-admin to mark one or more as 'featured'

[x] Dashboard + Controller (Related) {1d}

[x] Ordering at logic layer (date, views, rating, popularity of package, featured) {1d}

[x] date [x] views [x] featured

[x] 'Featured' related item with mechanism to fetch it. {0.5d}

[ ] UI within dashboard for sysadmins to check/uncheck feature status {0.5d}

[x] Changing auth for related.

Currently auth is based on deletion being possible by dataset owner and the related owner but it may be that a lot of the auth for related items should be configured to have more restrictive auth.

[x] Implement auth changes for publisher auth profile. { 0.5d }

[x] General issues

[x] Serverside validation of URLs

[x] Activity updates

[x] Make optional

[x] Related item only includes App, Idea, Visualization but not API | Post | Paper | News Article options as in original ticket #2204

[ ] UI Changes

[ ] General tidy up, fix form etc (icon for menu bar)

[ ] Highlight featured items py:if="related.featured"

[ ] Show either view count or popularity ${ related.view_count }

[ ] A user's related items should be on their home page

[ ] Related item plus sign is very poorly aligned within box

[ ] Add related items should be in drop down like for resources (?) to make it easy to add items

[ ] Hover should show a title that gives some information about what related are (not at all clear)

"Apps, Ideas, Visualization and other material related to this dataset"

[x] Related is a pretty terrible name in the UI - much better to have it called Apps, Ideas etc (perhaps title tag could even give more details e.g. "Apps, Ideas, Visualization and other material related to this dataset") -- btw i thought this was in #2332 but realize it wasn't (apologies for that!)

• Went with Apps & Ideas (for now)

[x] Documentation, documentation, documentation - AFAICT I can't see anything in v1.7 or master docs. I imagine this would be a short section like ​http://docs.ckan.org/en/ckan-1.7/commenting.html (and probably coming right after that in the ToC) which says

• What it does
• How to enable (and perhaps includes a nice-screenshot!)
• How to customize (e.g. can one customize the list of options of things one can create (e.g. can I set it to just be app and idea or ...)
• Also liase with Mark W re a blog post about this from the user perspective (a screenshot walkthrough ...)

[ ] Clearly mark the old extension as deprecated from v1.7 forward: ​https://github.com/okfn/ckanext-apps (I just met someone last week who was working on integrating this and had no idea it was replaced by something better in v1.7)

Currently lower case 'active' is the name of this page: ​http://s031.okserver.org:2375/user

Should be 'All Users' or 'Active Users'

Add generator=ckan tag in head of templates to identify site as generated by CKAN

title broken for

​http://s031.okserver.org:2375/en/dataset/new_resource/gold-prices

we have inherited the title from the extended page not from the base

should be 'add data - ckan' not 'add data - create dataset - ckan'

any idea how we fix this?

We get an error which should be improved the actual problem is this but not passed to user

⇝ KeyError?: 'ofs.impl'

View as: Interactive (full) | Text (full) | XML (full) Module ckan.controllers.storage:2 in auth_form view Module ckan.lib.jsonp:26 in jsonpify view

data = func(*args, kwargs)

Module ckan.controllers.storage:407 in auth_form view

authorize(method, bucket, label, c.userobj, self.ofs)

Module ckan.controllers.storage:200 in ofs view

StorageAPIController._ofs_impl = get_ofs()

Module ckan.controllers.storage:71 in get_ofs view

storage_backend = configofs.impl?

Module paste.registry:146 in getitem view

return self._current_obj()[key]

KeyError?: 'ofs.impl'

/home/seanh/Projects/ckan/ckan/ckan/tests/functional/test_related.py

ImportError? (cannot import name assert_regexp_matches)

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62935295/comments

Implementation:

​http://s031.okserver.org:2375/en/user/login

you know the score

see spacing between 'download' button and 'data api' button ​http://s031.okserver.org:2375/dataset/afterfibre/resource/66f5b980-06db-4ccd-a005-c18de4d68a4b

Luca De Santis reported these strings not marked in 1.7.1:

• /user/login.html ("Remember me" & "Sign in")
• /package/search.html and template/package/group/read.html (Tags, Resource Formats, Groups)
• /package/resource_read.html (the "No description" message)
• the "Add related item" dialog (some strings are i18n savvy, others not)
• the stats page (labels on the graphs)
• the history page (as far as the log messages are concerned).

lets check that they are marked in 1.8

we need to ensure that translations in .js files are ripper for translation

look at what is available as preference to creating our own

The about page for the DataHub? (thedatahub.org/about) could be improved. More importantly the default about page for a generic CKAN instance should be completely different - focus more on Open Data rather than the community hub idea (as this is more relevant for most installations) and remove specific references to tdh.

Where does the data go?

In a postgres database configured by the ckan.datastore_write_url config option which is a sqlalchemy url.

The user should have rights to create tables.

Whats the api like?

We will just implement it as logic functions like the rest of CKAN and will part of core. After that we may add some nicer api functions that use these but that is a secondary concern.

What are the initial logic functions?

• datastore_create
• datastore_delete
• datastore_show

What is the JSON input format for datastore_create

To begin with it can have the following keys. It is fairly consistent with Max Ogdens' gut servers. Except adds resource_id.

{
resource_id: resource_id # the data is going to be stored against.
fields: a list of dictionaries of fields/columns and their extra metadata.
records: a list of dictionaries of the data eg  [{"dob": "2005", "some_stuff": ['a', b']}, ..]
}

• The first row will be used to guess types not in the fields and the guessed types will be added to the headers permanently. Consecutive rows have to conform to the field definitions.
• rows: can be empty so that you can just set the fields
• fields are optional but needed if you want to do type hinting or add extra information for certain columns or to explicitly define ordering.

eg: [{"id": "dob", "type": "timestamp" }, {"id": "some_stuff", "type": "text"}, ...]. A header items values can not be changed after it has been defined nor can the ordering of them be changed. They can be extended though.

• Any error results in total failure!! For now pass back the actual error.
• Should be transactional

What json does datastore_delete take?

{
resource_id: resource_id # the data is going to be deleted.
filters: dictionary of matching conditions to delete
    e.g  {'key1': 'a. 'key2': 'b'}  this will be equivalent to "delete from table where key1 = 'a' and key2 = 'b' ".
    No filters (either not present or not defined) then delete the table. If we want truncate then add truncate: true to truncate the table.
}

What json does datastore_search take?

{
resource_id: resource_id # the data is going to be selected.
filters : dictionary of matching conditions to select
    e.g  {'key1': 'a. 'key2': 'b'}  this will be equivalent to "select * from table where key1 = 'a' and key2 = 'b' "
q: full text query
limit: limit the amount of rows to size default 100
offset: offset the amount of rows
fields:  list of fields return in that order, defaults (empty or not present) to all fields in fields order.
sort: comma separated field names with ordering e.g "fieldname1, fieldname2 desc"
}

Some free code: ​https://gist.github.com/3163864

What json does datastore_search return?

{
fields: same type as datastore_create accepts (i.e. with metadata)
offset: The same offset that was supplied in datastore_show
limit: The original limit
filters: The filters that were applied in data_show
total: # total matching records without size or offset
records: [same as data_create] # list of matching results
}

On error will return:

{
__error__: … sql error …
}

What types are allowed?

Aim to support as many postgres/postgis types that have string representations.

​http://www.postgresql.org/docs/9.1/static/datatype.html

​http://www.postgresql.org/docs/9.1/static/sql-createdomain.html

IDs

Each row in a table will be given an _id column which has an id generated by us which you can use in queries.

Other Features

Each row will store the _full_text index of all the data in the row. At some later point there will most likely be a way to index fields add constraints etc.

The settings > extras page for a dataset allows extra fields to be added, but only four can be added a time in a single atomic update. It would be helpful to have an 'add another' button on the form that can be used any number of times.

use ​http://leaflet.cloudmade.com/ properly to generate map backgrounds for demo instances

I'd suggest we add a generic "failed to login, invalid fields" in the error_summary block. Then more specific "this field is required" when a required field is left empty.

So in general, provide an error dict

Really we want this for CKAN Hosted & for creating demos.

Currently we're using ​http://leaflet.cloudmade.com/ - but by finding the area & taking screenshots. Would be better to interate with the app so that: a) this happens automatically and b) correct attribution will be visible on maps

to use on demos instead of the ckan logo & tag line.

Design here: ​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/63387712/comments#190457381 (First, text only one at the bottom)

With the text logo I'd like the tag line to be under the site name so some css like

float:left;clear:left;font-size:13px;line-height:0.1;

Also the site name can be larger if no tagline

commit daac54dbd856 I've added a new global g.header_css that sets the hgroup class to

'header-image' 'header-text-logo' 'header-text-logo-tagline'

I think maybe we should also have 'header-image-css' for the equivalent of the class="logo" in the link to allow/stop the image replacement

As per #2650 (on master/1.8 now)

If the resource has a description it isn't rendered as markdown ..