{22} Trac tickets (2647 matches)

The publisher index page should show an alpha paged list of publishers.

Write an importer that supports most well known variants of DCat in importing the data as CKAN packages. In particular, the following sources should be supported:

• CKANrdf generated exports
• opengov.se RDF (not really DCat)
• Sunlight Nationdatacatalog as harvested by dcat-tools.

Spec - see attached, which is a copy of the Pirate Pad from Lauren: ​http://piratepad.net/2C2iwiLDhd

Importer function is exposed to logged in users.

Alongside specifying file to import, user chooses between 'form schemas' - 'basic' and 'government'.

• Basic is as it is currently
• Government form has pre-defined Extra fields, careful validation of lots of fields, code formats/processes some field data for storage. e.g. searching for tag keywords.

All fields should be in step with the government custom input form.

For fields which have suggested values but the user can input his own value, this is achieved in the spreadsheet with two columns - the first being a drop-down and the second free text.

Estimate:

'created' attribute on User, Group and Rating

datetime object: default=datetime.datetime.now()

See example on revision

Requires migrate script

This occurs when CKAN is run with mod_wsgi (not under paster).

This is because of a unicode header being creating in rest.py.

Related to: ticket:90 Link to RDF version of CKAN data and ticket:163 (regularly convert CKAN data to RDF).

Convert ckan data to RDF with suitable ontology and put on semantic.ckan.net.

Object properties:

• id (uuid)
• apikey (uuid) -- migrated from the apikey table
• name (username = openid)

/account is being renamed to /user in the wui.

As designed by Sam Smith -> ​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/ Also to be used as the demo theme although I think they will diverge at some point. At the moment they should be implemented as one and the same.

Because CKAN's worth it!

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62268474/comments

Implementation:

​http://s031.okserver.org:2375/user/aron

This includes the tag page as well for now.

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62998445/comments

Implementation:

​http://s031.okserver.org:2375/en/tag

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61906351/comments

Implementation:

​http://s031.okserver.org:2375/dataset/

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62998362/comments

Implementation:

​http://s031.okserver.org:2375/en/user/register

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61696765/comments

Implementation:

​http://s031.okserver.org:2375

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61911712/comments

Implementation:

​http://s031.okserver.org:2375/group/test-group

Should have all supported extensions enabled, those that don't work should be disabled and ticketed. Also needs a customised homepage.

Implementation:

​http://s031.okserver.org:2375/dataset/edit/5b451cab-8671-4513-91c4-257784c1f484

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61752357/comments

Implementation:

​http://s031.okserver.org:2375/dataset/adur_district_spending

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/62308209/comments

Implementation:

(to be added once test server is working)

Discussion:

​https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61911712/comments

Implementation:

​http://s031.okserver.org:2375/group/new

We need a way to track changes on the registry (datasets edited or updated), globally and on a per country/publisher/etc. basis. RSS and Atom feeds are really popular, and after closing #191 and #1498 crating them from the search results should be fairly easy.

The following URLs are pretty self-explanatory:

http://localhost:5000/feed/registry.rss

http://localhost:5000/feed/country/AF.rss
http://localhost:5000/feed/publisher/worldbank.rss
http://localhost:5000/feed/organisation_type/10.rss

As we need to implement custom wrappers for countries, publishers, etc, we might as well offer a fully customizable feed, e.g.:

http://localhost:5000/feed/custom.rss?q=activity_count:[* TO 100]
http://localhost:5000/feed/custom.rss?publishertype=primary_source

Apart from the actual feeds, there will be a small amount of work at the template level to add the links to the suitable pages (and maybe a generic page showing all available feeds)

Should minify and concatenate JavaScript? and CSS as part of #2354

We should create a public AMI with CKAN pre-installed and configured such that users can easily create their own EC2 machine with a running CKAN to play with.

There are three phases:

1. [nils] Deploy an empty EC2 instance to become the CKAN image master instance
2. Install a CKAN and give it a standard configuration.
3. [nils] Create a AMI from the CKAN image master instance and publish it.

I am happy to do first and last. Who is installing and configuring CKAN?

Unfortunately AMIs are specific to region, architecture and storage type. We cannot maintain too many images, so a number of choices have to be made:

• Which distribution/version? Ubuntu 10.04 LTS
• Which architecture/instance-type? I suggest 64-bit/t1.micro
• Which region? I suggest us-east-1 and maybe eu-west-1
• Which storage type? EBS (way easier to make an AMI from than instance-store)
• Install CKAN from deb packages via mercurial/virtualenv? I assume the latter because the AMI is targeted to developers?

​1 ​2

CKAN looks a bit aged, it should be styled more modernly and some elements could be re-arranged:

• Collect user info in top bar
• re-add the logo to ckan.net
• Remove tags from main menu, replace with /sitemap.xml

Inpiration:

quora.com, github.com, Google Projects, Google Refine, etc.

CKAN.net or CKAN general theme?

To be decided. Suggest we start with ckan.net specific and then backwards integrate (?). Existing ckan.net theme repo:

​https://bitbucket.org/okfn/ckan-ckan.net

just passes template name

?template_name=...

As a

User

I want to

Create a new group via the WUI

Details

• locate at /group/new/
• You must be logged in to create a group
• Group creator is automatically given role of group admin
• Edit screen is same as edit screen ...

Storage represents something you can upload to (download is done in ticket:406).

• storage sub package with defined interface (maybe just use OFS)
• Only support uploading single files for the moment (though can do multiple files).
  • Relationship to build (ticket:306)
  • No support for syncing e.g. hg/git repos -- that's up to you as a user
• Upload command

At end of this we can deprecate Repository.

Having Repository encapsulate Index and Storage (without Storage being explicit) is not helpful and leads to duplication.

• branch from 1.7.1
• apply CKAN patches that are current part of installation instructions
• create i18n patch so custom i18n directories can be specified via a config option

Set up a Mediawiki to contain info on CKAN.net (community) conventions etc.

From Knud Möller:

when I try to delete a non-empty tag vocabulary via the API (through HTTP), I get an internal server error. Checking the logs, this turns out to be a consistency error raised by sqlalchemy:

Error - <class 'sqlalchemy.exc.IntegrityError?'>: (IntegrityError?) update or delete on table "vocabulary" violates foreign key constraint "tag_vocabulary_id_fkey" on table "tag" DETAIL: Key (id)=(21421955-7560-467c-af30-9f790b73e6ae) is still referenced from table "tag".

'DELETE FROM vocabulary WHERE vocabulary.id = %(id)s' {'id': u'21421955-7560-467c-af30-9f790b73e6ae'}

URL: ​http://33.33.33.10:5000/api/action/vocabulary_delete

The error makes sense, but I'm wondering if it would be useful to extend the API to also allow the deletion of non-empty vocabularies, possibly via a parameter (not sure what best practice in API design is). At the very least, it would be cool if the error message coming back in the response had more information in it.

Document all of the changes and usage for #1574 so that we can mark it 'done done'.

Make sure we document any setup issues/processes and the settings.py setup required for CKAN so that Mark can announce next week and it should be easily deployable/testable.

Internal (perhaps in README somewhere or wiki): how does process work, where does data end up and in what order etc

External: here's how to configure it and here's what you get ... (goes in main sphinx docs)

Current form stuff is not very good (uses formencode). Switch to formalchemy would improve this, especially on validation.

As an extra we could utilize the formalchemy pylons admin interface (pretty much for free).

Cost: 12h

Details

1. Replace htmlfill and formencode extract in controllers/templates with formalchemy
   1. Crude and simple
   2. Suppress unwanted fields (revision, state, all revisions) and sort out ordering to be similar to before.
   3. Sort out tag field with a special renderer
2. Validation - testing definitely required.
3. Fix up description and pretty css etc

No real benefit of having State object as opposed to a simple text field with enumerated values and this second option is both simpler and more flexible.

Main hassle here is that it requires migration in all projects that use vdm.

If we're going to do this we should do it sooner rather than later.

Cost: 2h (+ 1h for a conversion method)

(2008-10-29) convert existing directory on disk to a datapkg distribution (PythonDistribution? by default). This would be presented in the CLI as a convert command.

Details:

• Add a metadata file (setup.py), perhaps prompting for input
• List all existing data in that directory

Cost: 4h

• convert non-free fields to vocab fields (eg: interoperability_level)
• store values as JSON files in the ecportal repo
• translate output on dataset view page
• add tests

As part of the general documentation overhaul (ticket:1142) we (APS and RGRP) want to convert the current Sphinx docs into:

• An Admin Manual which is task-based and aims to cover everything a developer would need to know to set up and customize a CKAN install.
• A Reference Manual which is the primary source of reference for CKAN software - this includes API docs.

The current chapters of the Sphinx docs should be moved as follows:

• index.rst - copy some stuff from README.txt, keep short
• README.txt - split out installation - stop symlinking in, keep separate, write new intro in index.rst
• [NEW] install.rst - new file on installation
• [NEW] theming.rst - move over from wiki
• api - will stay (gradually move tutorials/getting started to wiki.ckan.net user guide)
• i18n.rst: internationlization. say we have x langauges. just set lang config option. if your lang not there yet see wiki page for how to prepare a translation file
• design.rst - REMOVE (can copy some over if you can be bothered to ​http://wiki.ckan.net/Vision (all philosophical stuff should go!))
• loaders.rst: move to wiki.ckan.net/Using_Loaders
• feeds.rst: move to User Manual (CKAN_Feeds or just Feeds)
• importer.rst: remove (don't even copy)
• forms.rst: remove (ping david read and ckan-dev asking for replacement documentation -- do we need this in core ckan -- do we want pure js ...?). Suggest we start with (stub out) ​http://wiki.ckan.net/Customizing_Forms
• forms-integration.rst: remove
• model.rst: leave but move to reference (only for core devs)
• load-testing.rst -> move to ​http://wiki.ckan.net/Load_Testing
• distributed.rst -> remove
• admin.html should consolidate with authorization.html (some of authorization.html is probably in ref section but howto in main manual) <-- high priority
• deb.html -> go into reference (query james gardner on list about moving this to wiki?)

Requirements

To provide a freemium service it is necessary to be able to provide differing levels of functionality based on the type of user (see #2550). These levels can be specific to the data hub but may require overriding functionality from core to provide these checks.

Initial implementation should focus on limiting access to datastore disk space.

Interface

These changes are currently only for the data hub and should be kept as much as possible within the data hub extension.

User Stories

• As a system component I want to find out if the current user has access to a feature (i.e. storage) and if so to what extent (xMb, xGb or unlimited).

• As a system administrator I don't expect to need to manage the levels of users or the features that this applies to.

Tasks

[ ] Clarification of requirements/analysis

[ ] Tests

[ ] Code

[ ] Model

[ ] API

[ ] UI

[ ] Documentation

Estimates

Find all places where date, times and time periods are rendered as strings in CKAN. Make them all go through the same code path (e.g. the helper function in helpers.py) with localization support.

For example, on the user index page right now there are time period strings like "less than 1 month" that currently do not get translated (and the way they're implemented doesn't look i18n-friendly).

ISO date format everywhere might be a good idea.

Need to consider local timezone issues.

Python standard library and Babel may have useful helpers for this.

Should the local timezone displayed be a global setting per CKAN instance? Or should it change depending on the location of the user who is viewing the page?

On registering a new account, if you try to use a capital letter in your 'Login', you get the unhelpful error message:

"The form contains invalid entries: Name: Url must be purely lowercase alphanumeric (ascii) characters and these symbols: -_"

What does 'Name' or 'Url' have to do with anything? It means 'Login must be ...'

But even better would be to make the login case-insensitive and allow capitals to be entered.

• So some kind of confirmation or approval by someone more senior?
• Alternative: just have an account request.
• Priority: 3

Provide configuration for reverse proxying that will correctly handle the mapping of a URL to a sub-folder (using X-SCRIPT-NAME)

• Analysis of the best solution [1d]

• Implement reverse proxying in tandem with #1653 [2d]

• Document and store the configuration files. [1d]

Related items (Apps & Ideas) have a collection of types, which are currently fixed. Some of these types are ambiguous, such as ideas in that they may not have a link - and if they do it is likely to be to a blog post - another existing type.

Whilst the URL is required, and we think this should stay required, we should also allow users to change the types found in Apps & Ideas to a shortened list that suits their requirements.