{22} Trac tickets (2647 matches)

Run CKAN with the publisher profile and organizations plugins enabled. Make an organization. Make a dataset and add it to the organization. Edit the organization and change e.g. its description. After saving the organization, all its datasets have been removed from the organization and now belong to no organization.

When editing a package one would like to be able to preview one's changes before saving.

I'm not sure if this was reported earlier, but I have found a minor bug in the "Add new package" form. As far as I can see this bug exists in v1.3.2 and also in v1.4.1a (that runs on ckan.net). This is a problem with adding new resources in "Resources" section. To reproduce the bug do the following:

1. Go to "Add new package", for instance ​http://ckan.net/package/new
2. Scroll down to "Resources" section.
3. Click on "Remove this row" link (red circle).
4. Say "Yes" to "Are you sure you wish to remove this row?"
5. Now there should be no rows. Only the header "URL* Format Description"
6. Click on "Add row to table"
7. All you get now is another row with table header "URL* Format Description". You will not be able to add any resources now. Solution is to reload the page.

The simplest solution to this bug will be to hide the "Remove this row" link if there is only one row.

Follows on from #826. We can now edit resource extras in the WUI (to some extent - see #978 for remaining issues) and we can view resource extras in the API, but we can't yet edit them in the API.

• code review with D.R. for ckanext-ecportal
• merge and test all multilingual code
• fix server issues: routing, Git access to CKAN repository

Bugs to fix:

• rename groups to publishers
• update home page layout (recent updates along side publishers)
• don't add 'en' to main menu links if english
• resource edit page, fix missing images
• change dataset 'settings' to 'edit'
• translate 'published by' drop-down list
• fix JS on 'add a dataset page'
• prepopulate 'published_by' when clicking 'add a dataset' from group
• chosen broken (dataset edit) if in different language

Implement the EC/ODP template for RDF to match the requested schemas.

Using the previous work on allowing for templatised RDF representations we should implement the proposed schema for EC/ODP.

We also agreed to allow upload of RDF as an extra (raised to top level in package schema) which will instead be used for the RDF template. On insertion extra CKAN triples will be added... initially this is just owl:sameAs although we're awaiting EC/ODP agreement on which triples we should add to show the CKAN link.

The database dump currently contains all Packages and their associated objects, even those that have been set to state=deleted. We should exclude these from the dump now.

Dump = paster db simple-dump-csv/json

reasoning

The dumps are designed for end-users to be able to run scripts on the mass of data. Since end-users don't see state=deleted packages then they shouldn't need them in the dump. In fact their presence in the dump probably confuses them.

Admins get the full database anyway in the backup pg_dump.

We only included them in the user dump because it was designed before use of state=deleted was established.

Time estimate: 2h

Make a robust way to make sure the drupal database is consistent with the ckan data.

reported as editing datasets incorrect

e.g. clicking on edit here ​http://s031.okserver.org:2375/dataset/ff takes you to create dataset page ​http://s031.okserver.org:2375/dataset/edit/ff

but this is the correct behaviour of a draft dataset

We need to show draft datasets correctly

proper tickets need making for the different issues after review of issues with them - who can see, where, admins and viewing, orgs too etc

Factor out current downloader into dedicated download subpackage, abstract interface and make it pluggable so we can have downloaders loaded as per type of resource to download.

• Also rename InstallCommand? -> DownloadCommand?

And we should support interactive retrieving of resources ...

Implementation Ideas

for resources in pkg.resources:
    # datapkg/hg 
    load_storage_handler(resource.format)

Support for download metadata (file size, file formats, version information).

See also: ticket:87 (multiple download links)

Dump and load JSON explicitly instead.

Quite a few of the current test dataset resources have their format incorrectly specified as text/html. Attempting to preview these results in the browser trying to automatically download the resource, so for now we will not preview this format.

On some occasions (apparently random), the documents indexed in a specific SOLR core get mixed with different site_ids.

E.g: We look for all documents in the testing.iatiregistry.org core, faceted by site_id. We would expect all documents to have site_id = iati_testing, but some of them have site_id = iatiregistry.org

​http://okfn-solr.fry-it.com:8080/solr/testing.iatiregistry.org/select?indent=on&version=2.2&q=*:*&fq=+state:active&facet=true&facet.field=site_id

<lst name="facet_fields">
<lst name="site_id">
<int name="iati_testing">265</int>
<int name="iatiregistry.org">255</int>
</lst>
</lst>

If we compare one of the records which disappeared from the "iati_testing" site_id in both the production and testing SOLR cores of the server, the records are exactly the same, including the indexed_ts property:

​http://okfn-solr.fry-it.com:8080/solr/iatiregistry.org/select?indent=on&version=2.2&q=id:97d1ab0d-b203-4757-8f4e-a0c84d2f759f&facet=true&facet.field=site_id

​http://okfn-solr.fry-it.com:8080/solr/testing.iatiregistry.org/select?indent=on&version=2.2&q=id:97d1ab0d-b203-4757-8f4e-a0c84d2f759f&facet=true&facet.field=site_id

Note that the response from the URLs shown may vary, as the testing site could have been reindexed.

There's no docs for setting up the internal analytics, nor what it provides.

Setup

Add: ckan.tracking_enabled = true to .ini file

Run: paster tracking -c dev.ini

Related links

​http://trac.ckan.org/ticket/2251

Add minimal, working IDatasetForm and IGroupForm example extensions to core, with tests.

The IDatasetForm example should use tag vocabularies (two birds with one stone)

The IDatasetForm and IGroupForm docs are not very good (and are somewhat spread around different doc chapters), fix them up, and reference the new working examples.

Tab Vocabularies docs should reference IDatasetForm example.

When using convert_to/from_extras() you have to remove any free extras from the form or it won't work, this needs to be documented (in the docstring maybe)

There have been recent changes to the schemas that IDatasetForm and IGroupForm use, make sure the docs are up to date.

There are 21 undocumented config options in CKAN, some of which are not mentioned in the config file template either:

ckan.admin.name ckan.admin.email ckan.default.group_type ckan.page_cache_enabled ckan.cache_enabled ckan.cache_expires ckan.extra_resource_fields ckan.extra_resource_group_fields ckan.storage.key_prefix ckan.storage.max_content_length ckan.feeds.authority_name * ckan.feeds.date * ckan.feeds.author_name * ckan.feeds.author_link * ckan.mail_from ckan.gravatar_default * ckan.plugins ckan.api_url ckan.auth.profile ckan.datastore.enabled ckan.tracking_enabled

There are also some options that are in the default deployment.ini even though they're deprecated:

ckan.async_notifier carrot_messaging_library ckan.build_search_index_synchronously

See email to ckan-dev from David Read: ​http://lists.okfn.org/pipermail/ckan-dev/2012-June/002447.html

It'd be best if the docs could be automatically pulled from the source into sphinx using autodoc, see #1358

Document deployment on tenforce test server.

• Create page on ​http://wiki.ckan.org/ about setting up a queue (this will merge into core docs when this is stable) - having a queue will now become a generic requirement for a bunch of extensions
• README with install instructions in ckanext-archiver
• Create an entry in ​http://wiki.ckan.org/Extensions
• Write a blog post

The documentation for setting up Apache uses sites-enabled instead of sites-available and a2ensite demo.ckan.net

Current we hide most of readme and then let users reveal it. Stop doing this and if necessary add a quick link down to resources section. (Maybe also rename resources to Data and Resources ...?)

Aside: believe I have mentioned this somewhere a month + ago but could not find the ticket.

Currently it is possible (v0.11) to have a PackageResource? object without a url. This should be impossible to do (either at creation time or via modification).

Based on the ideas in:

• DistributingChanges
• SyncingInstances

Merging the tickets #296, #297, #298, #299 into this one.

Related ideas include:

• Using dcat to exchange data
• Using couchDB to have a stub package record synchronised between multiple CKAN instances, each linking to the main record in the correct instance

This should be considered a wishlist item for the timebeing.

Just like we display download counts we should display view counts.

Currently use old "bullet-point" style. Should change to now standard style used on e.g. package search results page.

Cost: 0.5h

Split out from ticket:43

Superticket: #1343

Creation of accounts from OpenID causes large number of problems:

• No guarantee of other profile info
• Poor username (just from openid)

We therefore will:

1. Require creation of user account via register (no auto-creation of accounts via openid)
   • sidebar of register page should point to login page rather than openid signin

2. Permit association of an openid with an account ...
   • How: paste in your openid url (requires change to edit form to have this)
     • won't work for google with their weird openid urls
   • [future] Nicer way is to have login via openid while logged in (which association then happening in background)

3. Migration (for 1+2):
   • (?) Generate decent usernames for all existing accounts
   • As this is nontrivial suggest instead we allow editing of usernames (by account owner and sysadmins). This is useful in its own right and is a reasonable 80/20 solution.
     • This does present the problem of re-associating commits with the new username. Best solution to this would be to switch revisions to point to userid rather than username. See #1534

4. [optional] Change display_name to always be username
   • Change My Account to Username at top right of all pages (once we know usernames are short ...)
     • could do this straight away by truncating long usernames (e.g. truncate at 20 chars ...)
     • Also should we lose the icon?

Currently this raises an exception because there is no package/package_revision in existence at from_revision.

Exception raised doing this particular diff.

​http://ckan.net/revision/diff/sudocfr?diff_entity=package&oldid=a18b2520-09e0-44be-80dd-4fea94ffe7d1&diff=3934ff16-a3a7-4481-b390-72944903b945

...
Module ckan.controllers.revision:142 in diff
<<          if c.diff_entity == 'package':
                   c.pkg = model.Package.by_name(id)
                   diff = c.pkg.diff(c.revision_to, c.revision_from)
               elif c.diff_entity == 'group':
                   c.group = model.Group.by_name(id)
>>  diff = c.pkg.diff(c.revision_to, c.revision_from)
Module ckan.model.package:449 in diff
<<                              display_id = to_obj_rev.tag.name
                               elif obj_class.__name__ == 'PackageExtra':
                                   display_id = to_obj_rev.key
                               else:
                                   display_id = related_obj_id[:4]
>>  display_id = to_obj_rev.key
AttributeError: 'NoneType' object has no attribute 'key'

Seems to be to do with this particular extra key:

[<PackageExtraRevision id=ba1f3b3e-f61f-4924-bae0-ec3dd2163ae4 package_id=febe4da9-5874-4411-a7da-b497fdebf739 key=namespace value=http://www.sudoc.fr/ state=active revision_id=5ec8e730-c73d-4f1c-989b-2e015eb3d315 continuity_id=ba1f3b3e-f61f-4924-bae0-ec3dd2163ae4 expired_id=None revision_timestamp=2011-07-08 13:48:11.027260 expired_timestamp=9999-12-31 00:00:00 current=True>]

Is it because it was created between the from_revision and to_revision?

Need to establish procedure for automatically generating package names from metadata documents.

When a resource doesn't have a description, it is rendered as markdown from the description of the dataset.

If the resource does have a description it isn't rendered as markdown ..

See ​http://thedatahub.org/dataset/wikipedia-e3-timestamp-position-modification/resource/d883ab44-07f4-4992-800a-3e4bf5d53a96

I get this warning when viewing the home page:

event.layerX and event.layerY are broken and deprecated in WebKit. They will be removed from the engine in the near future.

CKAN: latest master Browser: Chrome Ubuntu

Minor, but worth checking?

In branch feature-1821-multilingual-extension, a new faceting datastructure was introduced. This makes the old one and related functions obsolete. These have been marked as deprecated for the 1.7 release, and should be removed for the 1.8 release.

• helpers.py:facet_items()
• facets.html:facet_sidebar()
• uses of c.facets (rather than c.new_facets)

test_authz.py appears to test in great detail some very specific additional authz (related to total site lock-down it seems -- introduced I think for hri project).

I think there are simpler ways to get total site lockdown (use external auth!) and this test is slow and delicate (e.g. depends on specific words in templates). Suggest removing. If we don't remove we should at least refactor tests for access to certain pages to use a proper method of testing (e.g. agreed html comments in each page) rather than being depending on the presence of absence of specific wording.

We need to deploy the new storage changes (and webstore) to thedatahub for further testing

• Need to figure out cache invalidation for CKAN
• Evaluate against an internal cache (i.e. memcached)

• deploy celery
• deploy QA and archiver tasks
• write up a blog post announcing QA on thedatahub