{22} Trac tickets (2647 matches)

Results (1301 - 1400 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1275 enhancement johnglover johnglover ckan-sprint-2011-09-26 closed fixed Move solr search extension to ckan core 1313413597000000 1319812967000000
#1576 enhancement rgrp rgrp ckan-sprint-2012-01-09 closed fixed Move stats extension back into core - 0.5d

Est: 0.5d.

Questions:

  • Why do this?
    • tiny extension with few dependencies - and really nice to have out of the box
    • trial for doing this on larger scale
  • Do we keep as extension (even if in core)?
    • Ans: Yes, keep as extension because:
      • Already set up that way
      • Cleaner
      • Easier to disable / enable
  • How do we integrate with main them (e.g. have stats link)
    • Ans: not sure (this is part of more general issue of how we update theme for varying changes elsewhere). Best answer is to have some widgetization in theme.
  • Hide ratings section (at least until we reinstate ratings #1598)
    • Ans: no, let's not bother (and having ratings there encourages us to do #1598 and/or find out whether people are interested in ratings)
1324317313000000 1327056070000000
#284 defect johnbywater rgrp v1.0 closed fixed Move tag/read/xxx to simple tag/xxx like package and group

Also put in redirect from old locations (see examples with plurals in routing.py)

Estimated cost: 20m

Actual cost: 40m

1270664552000000 1271852451000000
#947 enhancement rgrp closed duplicate Move the assignment of roles to a package on package creation to an extension

At the moment the assignment of user roles on package creation is done in setup_user_roles method in ckan/model/authz.py and is hard-coded.

This can be a pain to override (you can use a Plugin listening for package create events) and makes some things such as putting system in restricted mode more complex than it should be (see ticket:833).

An elegant solution would be to move this into an Extension this simplifies the code and make it easier for people override (just remove the default extension and plugin your own).

1296557478000000 1314031310000000
#932 enhancement dread dread ckan-v1.3 closed fixed Move to SQLAlchemy-Migrate 0.6

Basic steps from kindly:

  • changing the setup.py dependencies for sqlalchemy-migrate and sqlalchemy to their latest versions.
  • running the migrate script on the repository (or changing the repository scripts manually).
  • checking in both these changes at once.
1296156660000000 1297076710000000
#1136 enhancement kindly rgrp assigned Move to SessionExtension in vdm

When vdm was created there was no SessionExtension so we use MapperExtension for doing revisioning. Now that SessionExtension? exists we should use it. We can also follow the existing SQLAlchemy recipe: <http://www.sqlalchemy.org/docs/orm/examples.html?highlight=versioning#versioned-objects>

1305210855000000 1340632980000000
#1077 enhancement kindly rgrp ckan-backlog new Move to simpler vdm system

Option 1: 'Changeset' Model

See ticket:1135 for vdm ticket. This would involve a) moving to changeset in vdm b) doing the migration in ckan to support this.

Have developed a new "changeset" based model for revisioning in vdm.

Implementation

  • The main challenge with this change is schema and data migration

Every revisioned object has a revision_id and revision attribute.

Approximate algorithm:

Revision -> Changeset

for revtype in [PackageRevision, ...]:
    for pkgrev in package_revision:
        changeset = lookupchangeset(package_revision)
        ChangeObject(cset, (table, id), dictize(pkgrev))

Question:

  • does pkg include tags attributes or not? or we have to dictize, pkgrev, pkg2tagrev, and tag. Probably the latter.

Option 2: Simplify Revision Object Model

Just use a simpler vdm, see ticket:1136 (move to SessionExtension) and ticket:1137 (remove need for statefulness in vdm).

Discussion

Advantage of Option 1 versus 2:

  • Easier support for pending state and similar behaviour
  • No need to introduce new tables (and hence migrations) when making something revisioned (or not).

Disadvantages

  • Migration is required
  • More difficult to query revision history.
    • Could be addressed by having ChangeObject have separate cols for table name and id but would likely be more difficult.
  • Performance (?)
    • Have one big ChangeObject table to query when looking at changed objects rather than many revision tables.
      • Not sure this is a biggie as even with Revision model biggest revision object tables are probably on the order of the ChangeObject table

Conclusion

Implement Option 2 and leave Option 1 for present.

Option 1 includes Option 2 so it seems that that is required in either case (so we may as well with Option 2).

Option 1 requires significant effort (esp migration) so leave for present and then review the situation at some later date.

1302304464000000 1340034345000000
#1610 enhancement ross ross closed duplicate Move webstore to Postgres instead of Sqlite

The default backing store for webstore should be Postgres and not sqlite. This was agreed as part of the #1574 storage changes and on http://ckan.okfnpad.org/newstorage

1325587341000000 1325852472000000
#1462 defect amercader thejimmyg ckan-sprint-2011-12-19 closed fixed Multi-Instance CKAN Solr Search Issues

With the 1.5 package release, CKAN instances over-write each other's indexes. Adria has a fix.

I'd like to see a new CKAN 1.5.1 release with the Solr changes merged and the Solr site ID set up as part of the install ckan.site_id = dgu-os.okfn.org. I'd also like the ability to search on ID and to have the ID directly visible on the package page.

We need to decide how to do this upgrade without affecting existing CKAN solr indexes.

1321375470000000 1330083671000000
#3001 defect seanh ckan 2.0 new Multilingual plugin crashes CKAN on add dataset when some languages are default

Enable the multilingual plugins:

ckan.plugins = stats synchronous_search multilingual_dataset multilingual_group multilingual_tag

and set your default language to one not supported by the multilingual plugin, e.g.

ckan.locale_default = cs_CZ

now run CKAN and try to add a dataset:

File '/home/seanh/Projects/ckan171/ckan/ckanext/multilingual/plugin.py', line 141 in before_index

text_field_items+ default_lang?.extend(all_terms)

KeyError?: 'text_cs_CZ'

It doesn't matter what language you are viewing the site in in your browser, the default language setting in the ini file determines whether it crashed or not.

A number of supported languages are defined at the top of ckanext/multilingual/plugin.py. I think if the default language is not one of these it crashes.

I think this affects all versions of CKAN since the multilingual plugin was added so at least 1.7, 1.8 and 2.0

1350579048000000 1350579048000000
#2763 defect seanh ckan-v1.8.1 new Multilingual tests failing

test_multilingual_plugin.TestDatasetTermTranslation?.test_dataset_index_translation, test_multilingual_plugin.TestDatasetTermTranslation?.test_group_read_translation both failing for me on master

1343303819000000 1350303864000000
#87 enhancement rgrp rgrp closed duplicate Multiple download links

Multiple download links, including links to mirrors and multiple formats/versions

1248693302000000 1258470719000000
#189 enhancement dread dread v0.11 closed fixed Multiple download urls

See also: PackageResources which defines a 'distribution' as a package's payload (perhaps with metadata). In our case, at least within CKAN, the representation of the distribution will just be the reference to get it (plus some minor metadata). We are expanding this functionality to be any kind of 'resource' associated with a package since it is likely in some cases people will want to link to a file containing a part, but not the whole, of a package's data.

As a

User (package editor and package user)

I want to

(package editor) Associate multiple distributions with a package.

(package user) Easily have access to multiple distributions of a package should they exist.

Implementation

There is an object called 'Resource' with the following attributes:

  • url (text) (required in UI, but not in db table)
  • format (text)
  • description (text)
  • package (association) - a resource is always associated with 1 package

A package's list of resources will have an order which is preserved. The first in the list will be known as the 'default resource' (the presumption will always be that this is a 'distribution').

How to replace download url

download_url removed and replaced as a proxy attribute. Read returns the url of the default distribution. Write changes the url of the default resource. Ensures backward compatibility.

REST API has new attribute in package dictionary 'resouces' - an ordered list.

In WUI, replace download url in package read with a table of the distributions.

In the WUI package edit, be able to add/remove/edit/reorder distributions.

Example

Example part of package:

Distributions:

http://some.com/pollution.pdf PDF Original report
http://some.com/pollution.csv csv Original data
http://some.com/pollution sparql Linked version of data

Existing data to be migrated to new structure.

Cost: 4 days

1258386120000000 1265284961000000
#2824 defect danieljohnlewis demo phase 4 closed fixed Multiple of the same Flag Issues

There is some bizarre language flag issues. 1) Log in to http://s031.okserver.org:2375/ (I'm logged in as danieljohnlewis) 2) Visit the apps page: http://s031.okserver.org:2375/apps 3) Have a look at the languages drop down, flags look fine (although Deutsche does have an EU flag rather than the German flag.. but that's irrelevant) 4) Change filters to, say, type: Application and sort: Newest, and apply. Web address becomes: http://s031.okserver.org:2375/apps?page=1&type=application&sort=created_desc 5) Look at the languages drop down: Flags are largely incorrect, with more than one country having the same flag.

Please see the attached screenshot.

Note: this was originally tested with Google Chrome on Linux (Ubuntu).

1344505215000000 1344532640000000
#1480 enhancement dread dread ckan-sprint-2011-12-05 closed fixed Munge functions in util

munge_title_to_name, munge_tag v. useful for client software importing things into CKAN. I want it for ONS.

1321965837000000 1321978546000000
#379 task johnbywater johnbywater v1.1 closed fixed My dev task #1 1280325277000000 1280326496000000
#380 task johnbywater johnbywater v1.1 closed fixed My dev task #2 1280325304000000 1280326595000000
#378 story johnbywater johnbywater v1.1 closed fixed My user story #1 1280325186000000 1280326706000000
#1287 enhancement thejimmyg dread ckan-backlog assigned NAVL validation errors - Junk fields should be listed explicitly

When you create a package, but specify a key that is not allowed (e.g. 'relationships') then you get error message:

{"__junk": ["The input field __junk was not expected."]}

It should mention the actual key which is not expected. e.g.

{"relationships": ["The input field 'relationships' was not expected."]}

Kindly said that James' version of NAVL was better in this respect, so this might be best solved by moving to that.

1314203102000000 1330990459000000
#466 requirement dread johnbywater ckan-v1.3 closed wontfix Name of HTTP header for API key shall be configurable 1282308995000000 1294835610000000
#1742 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Naming of a new package/group can clash with a route

It is possible to create a group or a dataset called new, or edit, which will then be inaccessible as it is blocked by the existing routes with those names.

We should maintain a list of invalid names in the verification that happens behind the scenes in the UI.

1328116317000000 1331555193000000
#2752 enhancement toby ross demo phase 2 closed fixed Navigation highlight missing

When navigating to http://s031.okserver.org:2375/group on the demo server, the Group link in the navbar is not highlighted in the same was as the other links are when visiting those pages.

1343216582000000 1343222618000000
#2796 enhancement mark.wainwright ross new Need a datahub one-pager

A one-pager explaining what the datahub is and with howto/examples for new users. This would make it much easier to explain the value in using the datahub for storing data.

1343924916000000 1345129495000000
#2836 enhancement toby toby demo phase 3 closed fixed Need extra info for forms

like we have on the form here

http://s031.okserver.org:2375/harvest/new

you said it partly exists so maybe I just need some training

1344605931000000 1345023663000000
#2830 enhancement toby toby demo phase 4 new Need method to undelete groups

need controller action and front-end method

1344509408000000 1344547341000000
#2545 requirement ross ross opendatasuite 1 closed fixed Need to build DataGM site based on ODS

Requirements

We need to clone the demo ODS site for DataGM as soon as we have a mostly working demo so that the DataGM team can try it out and comment/feedback on functionality.

Interface

None

User Stories

None

Tasks

[ ] Deploy demo DataGM site.

Estimates

1340016640000000 1344337973000000
#1466 enhancement thejimmyg ckan-backlog new Need to support https login for multiple instances as part of the CKAN package install 1321375978000000 1328529062000000
#2802 enhancement aron.carroll aron.carroll demo phase 3 closed fixed Need to tweak the JSON formatting of the localisations

This is what we need the JSON to look like. Taken from the Jed documentation (http://slexaxton.github.com/Jed/).

{

      // The empty string key is used as the configuration
      // block for each domain
      "" : {
        // Domain name
        "domain" : "the_domain",

        // Language code
        "lang" : "en",

        // Plural form function for language
        "plural_forms" : "nplurals=2; plural=(n != 1);"
      },

      // Other keys in a domain contain arrays as values
      // that map to the translations for that key.
      // Gettext suggests that you use english as your keys
      // in case the key isn't found, and it can just pass
      // the value directly through.
      // Note: by convention, the 0-index location of the translations
      // is never accessed. It's just a thing, I guess.
      "a key" : [ null, "the translation", "the plural translations", ... ],

      // The plural form string is converted into a function
      // and the value that's passed into the gettext call
      // is passed into the plural form function. It's result
      // (plus 1) is the index that the translation comes from.

      // We're using sprintf interpolation on our keys so we can
      // then sub in the _actual_ values into the result.
      "%d key" : [ null, "%d key", "%d keys" ],

      // Contexts are keys that are just prefixed with a context string
      // with a unicode \u0004 as the delimiter.
      // You can use it for anything. Usually it's just for being content aware
      // in some way (e.g. male vs. female, product vs. category)
      "context\u0004%d key": [ null, "context %d key", "context %d keys" ]
    }

This looks like an appropriate python library to do the formatting. http://pypi.python.org/pypi/pojson/ but I'll leave to your discretion.

Finally an API endpoint that returns the appropriate request would save us making requests to the i18n directory.

/api/i18n/{locale}

For example if the current url is http://localhost:5000/pl/dataset then the client would request:

http://localhost:5000/api/i18n/pl

This should return the Content-Type: application/json and a 200 with an appropriate JSON body (this can be the language requested or an appropriate fallback). I think that if the fallback is the default language (in most cases English) then we just return a 204 No Content to save data.

1344248847000000 1344269446000000
#2948 enhancement dominik new Negative range breaks datastore backend

Recline allows negative ranges. The backend returns an error 500 for that which breaks the recline preview.

1349101854000000 1349101865000000
#181 enhancement rgrp jwyg closed invalid New 'stats' box on right hand sidebar

With up to date statistics such as:

  • total packages
  • total tags
  • total users
  • most active users
  • last update
  • etc.

Could also be nice to have a more dedicated stats page - with information such as graphs showing rate of packages being added and such like. See ticket:184

1257534385000000 1296339510000000
#2410 enhancement ross ross ckan-sprint-2012-05-29 closed wontfix New RDF serialization required VOID vocab for related datasets

Taken from http://lists.okfn.org/pipermail/ckan-dev/2012-May/002218.html


A few months ago, the RDF provided when dereferencing a dataset's URI was very useful. It described the datasets using many of the "right" RDF vocabularies (dcterms, void, etc).

The RDF that is returned now no longer has these nice descriptions. Instead, it seems to be a mindless transcription of the CKAN record.

For example, the older descriptions gave:

<http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>

void:subset [

void:target <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>, <http://thedatahub.org/dataset/geonames-semantic-web> ; void:triples 50 ; a void:Linkset

], [

void:target <http://thedatahub.org/dataset/dbpedia>, <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> ; void:triples 52 ; a void:Linkset

], [

void:target <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>, <http://thedatahub.org/dataset/govtrack> ; void:triples 52 ; a void:Linkset

] ; void:triples 130005 ;

but now it's simply:

<http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>

dct:relation [

rdf:value "130005" ; rdfs:label "triples"

], [

rdf:value "52" ; rdfs:label "links:dbpedia"

], [

rdf:value "50" ; rdfs:label "links:geonames-semantic-web"

], [

rdf:value "52" ; rdfs:label "links:govtrack"

] ;

My old copies of the dataset descriptions mentioned a "GoLD CKAN 1.0" that was credited for the nicer modeling.

Where did that go? Can we get it back?

Thanks, Tim Lebo

<http://semantic.ckan.net/record/91d2c0de-75a4-4bb6-b260-bc2946e1be8b>

dc:source <http://ckan.net/> ; opmv:wasGeneratedBy [

opmv:used <http://ckan.net/api/rest/package/farmers-markets-geographic-data-united-states>, <http://ckan.net/api/rest/revision/9c2545b8-ebf1-42bf-ac92-633ff5ceba3f> ; opmv:wasControlledBy [

a foaf:Agent ; foaf:name "GoLD CKAN 1.0"

] ; opmv:wasPerformedAt [

a time:Instant ; time:inXSDDateTime "2012-01-31T05:34:32Z"^^xsd:dateTime

] ; a opmv:Process

] ; a dcat:CatalogRecord ; dcat:dataset <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> ;

1337684531000000 1338193633000000
#1524 enhancement thejimmyg thejimmyg ckan-sprint-2011-12-19 closed fixed New Release of Latest MI Reports 1323172688000000 1323191733000000
#2957 task dominik new New datastore on postgres prior to 9.0

The GRANT SELECT ON ALL and ALTER DEFAULT PRIVILEGES are not available in Postgres prior to version 9.0.

1349292214000000 1349292214000000
#2732 enhancement ross ckan-backlog assigned New file upload functionality

We should simplify upload and storage of files, initially only to local storage with archiver eventually being fixed to archive data externally. WIP pad is http://ckan.okfnpad.org/uploads

Simplifying uploads

Currently uploads are too painful/difficult/fiddly to use and/or configure. We want to simplify uploads so that they are done directly to the CKAN server, without support for remote services (S3 etc) and/or the dependencies it introduces.

We want to fix:

  • File uploads themselves
  • Storage of uploaded files
  • Notification of the upload to other components

File uploads

Things file upload should do:

  • Allow sysadmin to disable
  • Allow auth'ed users to upload
  • Store whatever they send on disk, and store DB entry linking the file to the person
  • When creating the resource, the user should be able to choose from all of the files they have uploaded but not yet associated with a resource. This will allow for bulk upload and then a delayed association. Whenver a user creates a resource they either upload a file now, or see previously uploaded files.
Can we do the upload asynchronously and then associate the 
uploaded key with the resource before the save ? What happens 
if the user tries to submit before asymc upload finishes ? Should 
we delay them?

The upload workflow should look like...

  1. File upload should be a straightforward file upload with normal auth checks and normal processing of the posted data.
    1. When called via ajax then the ID of the newly created file should be returned,
    2. When called via WUI then it should also be given the url to redirect to after the file upload has been handled - the id will be passed as a query param.
  2. The resource save should check whether it has a file id and in that case updates the file object to point to the resource.

This should enable:

  • Separate file upload into a user's temporary store, either individually or as a batch.
  • Creating resources and simply choosing from previously uploaded, unassigned files
  • Adding files/data to a resource after the fact.

File storage

File storage should be local to the CKAN install, and not a remote service. Any archiving to remove storage providers should be outside of the main request.

File storage should:

  • allow moving data, a sysadmin should be able to move the storage root and change configuration and have the system continue running (i.e. don't store absolute paths).
  • provide maintainability, it should be easy to determine which old files are not associated with resources and thus can be cleaned up.
  • allow for collection of information (i.e. estimate of storate space used)
  • check whether there is enough space and handling the conequences cleanly
  • ensure files to be written only underneath its own root folder, checks should be made after any path generation that the file begins with the location of the file storage.
  • Have a configurable maximum accepted blob size during upload.
  • Should store what meta-data was provided with the upload, such as mimetype.

Somewhere in the DB we should store ...

ColumnNotes
idAn identifier
ownerThe owning user, who uploaded the file
pathThe path (from the 'storage root') to the file
sizeThe size in bytes of the file on disk
mimetypeThe mimetype of the file, as provided by the uploader
upload_dateWhen the data was uploaded
resourceThe ID of the resource it belongs to. A unidirectional relationship.
archived_urlThe URL where this file has been archived

Generating paths should try and separate the files, perhaps based on username of the owner, or some other mechanism to avoid a single folder full of files.

Notifications

We need to make sure that it is possible to notify other components within the system that an upload has taken place, or at least make it easy for them to be notified. The primary use case for this is to notify the component that will translate/upload certain formats to the data store.

We could do this based on the post-upload update to the file model (i.e. when we record the total received size of the file).

1343058789000000 1346663270000000
#3023 enhancement amercader amercader new New methods on IPackageController to provide access to the data_dict

Extension hooking into the edit and create methods of the IPackageController interface receive the package object. This may not include all the fields that came from the form. The new extension points will pass the validated data_dict so extensions can have access to it

1353603657000000 1353603657000000
#2523 enhancement toby toby demo phase 1 closed fixed New package metadata form needs creating

http://localhost:5000/dataset/new_metadata/dataset

Controller : package Action : new_metadata

template package/snippets/package_metadata_form.html

1339605176000000 1340368036000000
#3003 enhancement amercader ckan-v1.8 new New pg databases should be created with UTF8 encoding rather than system default

Just to track @tauberer patch on Github. Probably going to 1.8.1

1350639199000000 1350639199000000
#360 defect rgrp rgrp datapkg-0.7 closed fixed New sqlite-based DB index

In 0.5 and 0.6 have been more oriented to getting file indexes working to support the convenience of using packages on disk directly. However we now need to go back to having a better central/default index and the best model for this is a simple db index using sqlite (which is now supported in the standard library).

NB: this is a retrospective ticket (been working on this for a couple of months)

1277803174000000 1288004891000000
#3017 enhancement johnglover johnglover ckan 2.0 new New stable branch for the demo server 1352819088000000 1352819088000000
#2531 enhancement rgrp ckan-backlog new New state option: archived / deprecated

Deleted means things will get purged at some point.

Archived means they stay around but get hidden from search results and a big warning notice gets displayed saying this is archived / deprecated.

@richard cyganiak ...

1339750787000000 1339770649000000
#1378 enhancement zephod zephod ckan-sprint-2011-10-24 closed fixed New to TheDataHub? - StackOverflow style banner

Implement a banner across the top of the screen giving newbies a link to the about page, and making them feel welcome. See stackoverflow for reference?

1318247329000000 1324047835000000
#140 enhancement rgrp ckan-backlog new News section on front page

Have a news section (suggest as a sidebar item).

News section will link to latest 3/4 blog posts on CKAN from blog.okfn.org.

Details:

  • Suggest pulling via rss or similar.
  • Will want to cache this ...

Cost: 4h?

1254902541000000 1265625159000000
#2678 enhancement toby shevski demo phase 2 closed fixed No UI for deleting a related item

Needs to be a way for admins (owners of dataset) to delete related items. I suggest adding a button to the edit page

http://s031.okserver.org:2375/dataset/gold-prices/related/edit/d3249863-040a-4f55-a5d8-c86697d0b7b8

1342436087000000 1342685932000000
#1353 defect nickstenning ckan-v1.5 closed fixed No UI to remove resources

I have no idea whether this was a deliberate decision or not, but there is a total absence of any UI with which to delete resources from the currently deployed version of thedatahub.org.

1316729765000000 1317075904000000
#946 defect pudo dread ckan-v1.3 closed fixed No default system rights

CKAN currently comes out of the box being uneditable. We'd like to encourage people to have open CKANs and therefore by default init the db with this right:

visitor -> is editor on -> System system
1296499922000000 1296833383000000
#1801 enhancement dread ckan-backlog closed fixed No links to password reset

You can reset your password (#1186) but you have to know the URI (/user/reset) - there is no link to this page!

1329405290000000 1343144718000000
#694 bug wwaites dread closed fixed No postgres tools for current version

Database for all CKAN instances upgraded to Postgres 8.4, but none of the eu machines were upgraded with the tools necessary to administer them.

1286977052000000 1287087916000000
#1150 defect johnglover timmcnamara ckan-sprint-2011-10-28 closed fixed Non-ASCII chars prevent data preview

Characters outside of ASCII range are not supported within data previews.

Steps to reproduce:

  1. Visit http://ckan.net/package/kivele2010
  2. Click on [preview] for any of the resources
1306019914000000 1311774141000000
#2635 enhancement dread new Non-destructive SOLR reindex

You can't run the search-index reindex on a live server because it will give us bad results for 2 to 3 hours while it runs. Can there be an option that doesn't delete the entire index at the start?

Instead it could just delete any items that don't exist any more, then delete them and regenerate them one by one. So the total number of datasets doesn't change much.

1341829394000000 1341829394000000
#2483 enhancement markw ckan-v1.9 new Non-local resources should not have Download links

At present, a resource which is just a URL link to an external resource has a 'Download' button on the resource page. This gives the misleading impression that the resource is stored locally.

This is related to another small UI issue: I think the URL of a resource should be much more prominent, not buried in the 'Additional Information' table.

Suggested fix:

  • Put the URL prominently at the top of the resource page (above the preview) (and make it link to the resource)
  • Disable the Download button unless the resource is stored locally.
1338468734000000 1339771043000000
#2872 defect seanh ckan-v1.8 closed fixed Non-open licenses are marked as open

Set a dataset's license to 'Other (Not Open)', the dataset will have an Open Data icon next to it

1345124135000000 1350304179000000
#2416 enhancement toby aron.carroll demo phase 5 accepted Normalise resource/data types

Currently we have far too many types that are essentially the same format. The new demo theme is using icons for common types. So we need to normalise the common formats into pretty strings. e.g. application/json, JSON, .json and json all should be output as “json”

See the following Basecamp thread for UI examples and discussion.

https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/61906351/comments#comment_179681056

1337792946000000 1342617802000000
#1701 enhancement amercader kindly ckan-sprint-2012-02-06 closed fixed Normalize character encoding for ckan search.

Make sure accented characters are normalized when indexed and when searched for.

1327419050000000 1330085360000000
#1702 enhancement amercader kindly ckan-sprint-2012-02-06 closed duplicate Normalize character encoding for ckan search.

Make sure accented characters are normalized when indexed and when searched for.

1327419369000000 1327419922000000
#1387 defect toby johnglover ckan-v1.8 closed fixed Not specifying a protocol in dataset url causes broken link to be served on dataset view page

If a user enters a url with no protocol (eg: www.data.gov.uk) in the 'url' field of a dataset, a broken link is created on the view page.

CKAN should probably either put in a http:// if no protocol is given, or validation should fail.

1318428893000000 1343042311000000
#1244 enhancement dread assigned Notes field carriage-returns converted to CRLF

When you edit a package in the web form, if the notes field had \n as the End Of Line symbol, it gets lost when you preview or save the package, and the notes field is displayed all on one line.

This can be seen when editing annakarenina (as created by 'paster create-test-data'). The diff shows for example:

- Some test notes
+ Some test notes
?                +

but it would more clearly be shown as:

- Some test notes\n
+ Some test notes
?                ++

This is a significant problem with DGU, since a lot comes in via the API.

It's not clear what we should do about it. We could standardise on \n or \r\n when the form submission comes in. Do different browsers on different platforms do different things with EOLs?

Analysis

Displaying the package: the Markdown processor respects both EOLs when displaying the field, putting each line in a <p> tag.

Creating the package edit form: placed into <textfield>.

Browser displaying package edit form: <textfield> displays \n and \r\n as EOL. But \n\n gets compressed to one EOL. But on submission, both are returned as \r\n.

Receiving the edited package: Somewhere along the line the EOL gets converted to \n\n.

1311689456000000 1340191253000000
#367 enhancement dread dread closed fixed Notfication monitor

Runs on the commandline and prints out notifications.

1279303310000000 1279303693000000
#939 enhancement rgrp closed fixed Notification bar at top of site shown to new users pointing them to simple instructions

On http://getthedata.org/ or stackoverflow there is a pop-up bar for new users that give them some simple instructions.

Talking with users it is clear that some people are not clear how ckan.net (or other sites) work and what they are and are not 'allowed to do. This could help make this better.

Implementation

  • Pop-up bar (based on a cookie or just being not logged in?) - uses ticket:938 (message flashing)
  • FAQ/instructions page (use the new wiki?)

Depends

Estimate

  • Cost: 1h
1296399008000000 1323171158000000
#323 enhancement dread dread v1.1 closed fixed Notification message

Which events to notify on

Listed by domain object, these are the notification message 'change types' that will be sent:

  • Package
  • PackageResource

Also it is clear that it could be useful to know when db-wide maintenance is carried out:

  • db - 'clean', 'rebuild' (db is wiped and replaced with new data), 'upgrade' (migration)

Ignored domain objects

These parts of the domain model will not carry notifications as no use case has been identified for them:

  • Revision
  • Group
  • Tag
  • Rating
  • User - list of users is sensitive info
  • Relationships - complicated
  • Authz - complicated and sensitive info
  • License - change of a license's metadata is a question for the 'license service'

Message format

A notification message's header contains the routing key, identifying the object type. The client is probably interested in the object (all use cases so far), so it makes sense to send the object in the payload. This should be the JSON-encoded dictionary exactly as provided for the object's REST Entity.

For the 'db' notifications there shall be no payload.

1274723333000000 1278578841000000
#489 story wwaites johnbywater closed fixed Notify RDF service of model events 1282426051000000 1294416189000000
#487 story johnbywater closed duplicate Notify SOLR service of model events 1282425910000000 1291639404000000
#2826 defect toby danieljohnlewis demo phase 3 closed fixed Null group image

Problem: Example group ( http://s031.okserver.org:2375/en/group/example-group ) has a broken image, image not found. Link either incorrect, or no image set, if no image set then shouldn't it show a default image? Expected: There should be an image.

1344505546000000 1344528004000000
#2547 enhancement shevski ross opendatasuite 2 assigned ODS Initial data sets

Requirements

The ODS demo site will need data adding, initially as fixtures but it would also be useful if we started evaluating datasets that we can ship with ODS installations (at least in the UK) from places such as DGU and ONS.

May wish to create a ticket for making sure the datasets within the system are reset every X hours. Perhaps.

Interface

None

User Stories

  • As a new system administrator for an ODS instance, I don't want to have a site devoid of any data. Geographically relevant datasets would be welcomed.
  • As a bizdev person I would like to be able to demonstrate how ODS works with real datasets.

Tasks

[ ] Identify relevant sources for datasets

[ ] Pick datasets

[ ] Set them up for import

Estimates

1340016906000000 1340705614000000
#2546 requirement ross ckan-backlog assigned ODS Managing homepage content

Requirements

Require the ability for users to control some level of content that is visible on the home page of their ODS installation. This may be through RSS/Atom feeds (see #2234) or another mechanism but should result in admins being able to change blocks of text on their homepage.

This should not be configuration, but accessible through WUI.

Interface

None

User Stories

  • As a system administrator I want to have control over content displayed on the front page beyond featured/popular items.

  • As a system administrator I don't want to manage content through having to write an extension.

Tasks

[ ] Analysis

Estimates

1340016842000000 1346663437000000
#509 story dread dread closed invalid ONS data imported daily 1282665870000000 1291734435000000
#429 requirement dread ckan-v1.2 closed fixed ONS data is imported daily to DGU 1281703294000000 1285757274000000
#873 defect dread dread closed fixed ONS loader assigns packages to same package even when dept name added manually

e.g. national_child_measurement_programme had no department but "Information Centre for Health and Social Care" for agency, and TNA inferred it should be under department DoH. But now the next ONS update to this series expects a blank department, so creates a new package national_child_measurement_programme_.

1292437253000000 1292939274000000
#872 enhancement johnbywater dread closed fixed ONS loader assigns packages to same package even when dept name changes

Solutions: ResourceSeriesLoader? allows synonyms for values.

1292424112000000 1292957110000000
#880 defect dread ckan-v1.3-sprint-2 closed fixed ONS loader tripping up on deleted packages

Deleted packages with the same title and extra fields as an active package trip up the loader. It finds them in the search, but due to ckanclient not using the apikey when you 'get' the package to examine it further, it causes an exception.

1292939379000000 1292945137000000
#208 enhancement dread rgrp v0.11 closed fixed Obfuscate author email and maintainer email field

Obfuscate author email and maintainer email field.

Use: webhelpers.html.tools.mail_to e.g. webhelpers.html.tools.mail_to('mike@…', encode='javascript')

Do we want to? Surely only public addresses should be on CKAN in which case why obfuscate?

1260203938000000 1265891070000000
#2548 enhancement kindly ross datahub-july assigned Object ownership for groups/package

Requirements

We need to be able to easily determine who the owner of a dataset or group is. Datasets and Groups should have an Owner, who may change over time but is a specific user within the CKAN instance. It should be easy for CKAN components to determine the user and for the initial version we should ignore the can of worms labelled 'ownership transfer'.

At this point migration is likely to be the biggest issue, and would suggest that it is acceptable that the last user to edit a dataset be set as the current owner.

More tickets should arise as a result of this work where we may be able to optimise some queries to use the new feature.

Interface

None

User Stories

None

Tasks

[ ] Analysis/Clarification?

[ ] Tests

[ ] Migration

[ ] Code/Schema? changes

[ ] Documentation

Estimates

1340017331000000 1340706539000000
#2209 enhancement ross ross ckan-sprint-2012-03-19 closed fixed Object->RDF mapping

Super is [1649]

We need to provide customisable mappings between CKAN models and RDF. This needs to be easily configurable for different clients and should take into account objects related to the model being mapped.

Current contenders for impl are:

  1. ordf.org's Object Description Mapper (which uses OWL)
  1. RDFAlchemy, although this looks like it may result in fixed mappings
  1. A custom scheme with a simple json file
  1. A template based approach using Genshi templates?

Approach is to get a mapping complete of the attributes directly in the package class, and then provide mappings for related objects (tags/resources etc).

We'll then need to be able to publish the RDF to a triplestore.

See http://ckan.okfnpad.org/rdfmapping for further analysis.

1331027277000000 1336566051000000
#768 task johnbywater johnbywater ckan-v1.3 closed duplicate Obtain example ArcGIS CSW server 1288178134000000 1294409308000000
#529 task pudo rgrp closed fixed Obtain sample metadata 1282899161000000 1283536554000000
#1173 enhancement amercader amercader pdeu-1 closed fixed Offer a DCat representation for packages in the API

/api/rest/package/foo.rdf should return a DCat representation of the package. To create it, we will use the functions in ckanext-rdf.

1306939639000000 1307615271000000
#1452 enhancement dread dread closed wontfix Offer detected browser language, rather than auto-switch

There are issues with setting the site's language according to the browser detection:

  • Many browsers have the language setting incorrectly set. e.g. people who download US versions of Mozilla, pirated US copies in the far east.
  • Although thedatahub.org has a worldwide audience, many CKAN instances are designed for a particular country's citizens. So Open Berlin should probably be in German as default for all.

I suggest the site should have a default language saved in the config. The browser language *should* be detected, and that prompts a flash message offering to change to that language. And if you change language that is saved in the cookie (as we currently do).

Then do some testing to see if this suits people.

1320682652000000 1340190587000000
#2619 enhancement seanh seanh ckan-v1.9 assigned Omit private datasets from public activity streams

Activities about private datasets should not appear in public activity streams.

I don't think you want to actually purge the activities from the db, because you might still want them to appear in private activity streams.

I do think that when a dataset goes private all its past activity should go private, because I imagine that users are going to want to hide everything about the dataset and not have any past activities 'leaking out'

I don't think you want to consider whether the dataset was private when the activity happened, rather if a dataset is private now then all its past activities are private (and the simplest thing would be to say that if a dataset is public now then all its past activities become public as well, but is that a privacy concern?)

The easiest way to implement this is going to be by modifying the *_activity_list() action functions in get.py, after they pull their activity lists out of the db they should pass them through a function that filters out stuff about private datasets.

An activity about a private dataset is one whose object_type is 'dataset' and whose object_id matches the id of a private dataset. You should also check the object_type and object_id of all of the activity object's activity detail objects, if any of those match a private dataset then mark the whole activity as private.

Currently all activity streams are public so should have all private datasets filtered out from them, except for the dashboard activity stream which is private to the individual user. In this case private datasets that the user has permission to see should not be filtered.

1340884140000000 1351531137000000
#2658 enhancement johnmartin aron.carroll demo phase 5 closed invalid On Demo change 'Groups' to 'Publishers' everywhere

Can we do this as a localisation?

1342008495000000 1352205894000000
#3009 enhancement seanh seanh ckan 2.0 closed fixed On-site notification of new dashboard activity

We want a "bubble" with a number in it to appear in the top-right of the site next to your user name when you're logged in, telling you how many new activity stream items you have in your dashboard activity stream.

There's a github issue for this here:

https://github.com/okfn/ckanext-pdeu/issues/11

1351615762000000 1353949938000000
#840 enhancement dread dread ckan-v1.4-sprint-6 closed fixed On/off switch for ETags cache
  • Use config variable to switch ETags caching on or off. Consider joining in with 'cache_enabled'.
  • Default setting for (all) caching should be off.
  • Needs documenting in configuration.rst
1291308727000000 1302694133000000
#2408 enhancement aron.carroll markw demo phase 3 closed fixed Only 4 'extras' can be added at a time

The settings > extras page for a dataset allows extra fields to be added, but only four can be added a time in a single atomic update. It would be helpful to have an 'add another' button on the form that can be used any number of times.

1337683639000000 1343657014000000
#1028 defect dread dread ckan-v1.4-sprint-3 closed fixed Open redirect in locale setting 1299759883000000 1299760360000000
#562 defect pudo ckan-v1.2 closed fixed OpenID Login fails using Yahoo and Google

reported by LOD folks.

subsidary bug: error messages from OpenID providers are not shown on the form return page.

1283941119000000 1284063574000000
#1830 enhancement dread dread ckan-sprint-2012-03-05 closed fixed OpenID disablable

Allow OpenID to be disabled as a login option.

Currently (CKAN 1.6) there is a config option openid_enabled which stops the login option being displayed, but:

  • In the User edit form you can still add your OpenID
  • The repoze.who.openid middleware is still required - can it be avoided?
1330002137000000 1330089912000000
#940 defect pudo rgrp ckan-v1.3 closed wontfix OpenID login via google for same user generates 2 different openid urls

This has been noted twice in the last couple of weeks.

Checking around my guess is this is due to someone logging in to www.ckan.net and ckan.net. Google generates openid based on the RP (requesting party) which is the requesting domain url unless the openid.realm is set. See:

To fix this we should set the openid.realm (not sure how atm).

If this is not possible an alternative is to make sure we redirect all users to a single site url (perhaps using ckan config option site_url, or doing this in apache).

Cost: 1h

1296401521000000 1297071748000000
#1662 defect dread closed wontfix OpenID not compatible with mounting CKAN at non-root URL

Mounting CKAN at a non-root URL was made to work properly here: #1659

Unfortunately OpenID doesn't play nicely and would require some work to get working.

1326730366000000 1326730414000000
#1424 enhancement dread ckan-backlog new Openness notice should be clearer

ckan-discuss discussion suggests changes to the 'openness' indicator ( http://lists.okfn.org/pipermail/ckan-discuss/2011-October/001786.html )

Dataset view page:

  • If there is an explicit but non-OKD compliant license, such as CC-BY-NC, then this should be stated explicitly, perhaps: “This dataset is Not Open. License: Creative Commons Attribution Noncommerical. This is not an open license as it does not meet the Open Knowledge Definition.”
  • If the license is marked as “Other::License Not Specified”, then this should be stated explicitly, perhaps: “This dataset is Not Open. It is published without an explicit license, the publisher reserves all rights to the dataset.”
  • 3. If the license field was left empty by the contributor of the Data Hub record, then again this should be stated explicitly, perhaps: “This dataset is Not Open. The license of this dataset is unknown or unspecified. Start an enquiry on IsItOpenData? »
  • There is a bug so that non-open licenses doesn't have an openness notice.
  • If downloadable resources are not available, this should not affect 'openness' - check this has been removed.
1319648089000000 1319648089000000
#899 enhancement rgrp dread closed fixed Optimise check_real_change

There is no need to re-query the database to detect if there has been a changed as sqlalchemy knows this. It saves about 15 secs in the ckan tests. https://bitbucket.org/kindly/vdm/changeset/0050d023ca4e

From David Raznick

1294659583000000 1294916615000000
#1644 enhancement shevski ckan-backlog new Order default dataset page by most downloaded resources on thedatahub

Instead of alphabetically as we do currently, alternatively by most viewed datasets

for http://thedatahub.org/dataset

1326393542000000 1326393542000000
#1054 defect kindly dread ckan-v1.4-sprint-4 closed fixed Ordering of resources

Changing an old resource and creating a new resource on the end results in the old resource moving to the end in the ordering.

This breaks tests:

  • (ckanext-dgu) ckanext/dgu/tests/ons/test_ons_loader.py:TestOnsLoadBasic.test_fields
  • (ckanext-importlib) ckanext/importlib/tests/test_loader.py:TestLoaderInsertingResources.test_0_reload

You can make a ckan test break with this patch:

diff -r e6643cf1324c ckan/tests/models/test_resource.py
--- a/ckan/tests/models/test_resource.py        Wed Mar 23 13:25:52 2011 +0000
+++ b/ckan/tests/models/test_resource.py        Wed Mar 23 19:22:35 2011 +0000
@@ -297,6 +297,8 @@
                'url':self.urls[1], 'format':u'OTHER FORMAT',
                'description':self.description, 'hash':self.hash,
                'id':original_res_ids[2]},
+            { #new
+                'url':'new'},
            ]
        pkg.update_resources(res_dicts)
        model.repo.commit_and_remove()

There seems to be a problem with vdm creating a replacement Resource for the old resource - because it has a duplicate position it is put to the end by the SQLAlchemy ordering_list function.

1300969236000000 1301305615000000
#2722 defect aron.carroll shevski demo phase 3 closed fixed Ordering or search of datasets of a group brings up 'no results'
  1. go to a group, e.g. http://s031.okserver.org:2375/group/data-explorer
  2. Order by anything, e.g. choose 'popular' and click go
  3. "sorry no datasets found" message appears instead of ordering the group datasets in wanted order

http://s031.okserver.org:2375/group/data-explorer?q=&sort=views_recent+desc

1342948854000000 1344510010000000
#2917 enhancement johnmartin ross ckan 2.0 new Organization admins can delete themselves

Organization administrators can delete themselves from the user management pages. We should disable this on the front-end as well as the back-end.

1347451843000000 1347455636000000
#2970 defect seanh ckan 2.0 new Organization and group member links use id not name

e.g. it the 'Members' button links to /organization/members/0a44... instead of /organization/members/foobar

1350296531000000 1350296531000000
#2623 enhancement ross seanh closed invalid Organization join/authorization buttons weirdness

Enable the organizations and organizations_dataset plugins. Start CKAN. Login and create a new organization. Logout. Login with a different account that is a sysadmin. Go to the organization you just created with the other account. There are Edit and Join buttons. Click on the Edit button, and the Join button changes to Authorization.

It should probably be an Authorization button right from the start if you are a sysadmin.

1341220902000000 1341232433000000
#2967 defect seanh ckan 2.0 new Organization members edit page reloads after demoting self

Edit an organizations members page and demote yourself from organization admin, after saving the members edit page reloads even though you are no longer an admin and should no longer be able to access this page.

1350296227000000 1350296227000000
#2255 enhancement ross ross ckan-sprint-2012-04-30 closed fixed Organizations (Privacy + Ownership)

[x] The publisher_form extension and the 'publisher' component of the system will be renamed to organizations to both make its functionality clear, and to differentiate it from the publish auth profile.

Ownership of datasets is important and this ticket will enforce membership for a dataset to a single individual organization. The workflow for adding a dataset to a publisher/organization will stay the same but there will be no mechanism for adding existing datasets to an organization from within the organization forms.

[x] New datasets can be created from the organization form, in which case the organization will be set in the dataset form dropdown (with privacy set to private - see below)

[x] Datasets created from direct access to /dataset/new will have a drop down showing all of the organizations available to the current user.

[x] When editing a dataset the organization can only be changed if the current user is the admin of the current organization, and only then to one of the organizations available to the current user.

Only Organization admin can see the edit Organization form.

[x] Privacy will be enforced by changing the dataset form to have a public/private switch next to the organization drop-down which will be set as the capacity with which the dataset is added to an Organization. Private means visible only within the Organization, and should be the default.

[x] Search will be affected as the public/private status of a dataset will need to be added to the schema

[x] Only public datasets showing in site search, and private showing in organization dataset search (if user is within the organization).

[x] Make sure resources are restricted in visibility if the parent dataset is private

[x] Documentation should be updated to reflect these changes.

[x] Verify, then add, then re-add to documentation.

  • A User joins or creates an Organization
  • If creator of the Organization User becomes administrator of the Organization otherwise they become a Member.
  • Creation of new Organization must be approved by a System Administrator
  • New Members must be approved by the Organization Administrator
1332417218000000 1334663670000000
#2846 defect seanh ckan-v1.8.1 new Organizations allows you to set an organization as the parent organization of itself

that doesn't make any sense

1344860536000000 1350303844000000
#2624 enhancement ross seanh ckan-v1.8 closed fixed Organizations breaks dataset tag editing

Enable the organizations and organizations_dataset plugins, login to ckan, create a group, create a dataset and add it to the group. Edit the dataset, add some tags to it. The new tags are not saved when you submit.

Disable the organizations and organizations_dataset plugins and tag editing works again.

1341226411000000 1341831716000000
#2309 enhancement ross ckan-v1.8 closed wontfix Organizations review with SH

Review organizations code with Sean and explain the key parts.

1334591679000000 1340623966000000
#2939 enhancement toby toby new Orgs are groups

holder for ticket number

1348564034000000 1348564034000000
Note: See TracReports for help on using and creating reports.