{22} Trac tickets (2647 matches)

Results (601 - 700 of 2647)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#811 defect cygri ckan-backlog new Extra field editing form layout breaks when there are long field names

The layout of the editing section for extra fields breaks when a field name is slightly too long. Field names jump over to the right. See http://ckan.net/package/edit/dbpedia for examples.

1289994812000000 1323170289000000
#812 defect cygri ckan-backlog new Package edit form only allows three extra fields

Rationale

The package edit form is restricted to three extra fields. To enter more than three fields, one has to save the package and hit edit again (or hit preview).

Implementation

A mechanism similar to the one for resources (where you can add lines as you go) would solve this. So, have a button that adds more extra field rows via JS. (Extra fields don't need up/down buttons that the Resource table has)

Nice to have: a blank field is added when you tab from the last filled-in field in the table.

1289995010000000 1311176917000000
#818 requirement cygri ckan-backlog new Rethinking the author and maintainer fields

The semantics of the Author and Maintainer fields are really unclear at the moment. This leads to very inconsistent usage. Also, perhaps Name and Email are not the only fields that are needed for a contact.

Here is a table that shows the current usage of these fields in CKAN: http://richard.cyganiak.de/2010/ckan/ckan-ppl.html

We note several problems:

  • Author and Maintainer are often the same
  • Author and Maintainer are often used interchangeably
  • People really want to specify URLs for the contacts and stick them into random places because there is no field for it
  • Multiple comma-separated names in a single field

I'm not sure what to do about this, but a redesign is necessary in my opinion.

Some ideas:

  • Remove the maintainer field?
  • Make really clear that Author doesn't refer to the metadata on CKAN, but to the original data
  • Add an “author URL” field?
1290003524000000 1339774621000000
#819 defect cygri ckan-backlog closed fixed Tag autocompletion widget broken

The widget for tag autocompletion is broken in various ways.

For example, if I edit a package that is tagged "music", and just tab through the form fields to get to the Author field, then the widget changes to "industrial-music" as I tab through it.

Or if the tag is "foo bar" and I hit alt-right to jump to the end of the text field (with the intent of adding a third tag), then the contents change to "foo barbecue".

1290004121000000 1323171435000000
#837 enhancement rgrp ckan-backlog new CKAN integration with freebase gridworks / google refine

Thread: http://lists.okfn.org/pipermail/ckan-discuss/2010-November/000718.html

Scenario 1

  1. User installs Refine and CKAN extension for refine
  2. On booting refine and asked to load data they can choose from any data package on CKAN.net (or any other CKAN instance)
  3. They edit the dataset on Refine
  4. On save (or perhaps as a separate option) they are prompted as to whether they wish to sync the dataset back to CKAN (either as a new package or as a new resource on the existing package)

NB: for the dataset sync back some form of "CKAN" storage would be required (we already have storage.ckan.net running but a closer integration would be required)

Scenario 2

  1. User visits a package on CKAN.net (or another CKAN instance)
  2. There is a button on the page "View and edit this dataset in Google Refine"
  3. Click button -- ask them if they have Google refine installed
    • Yes: instructions for loading dataset into refine
    • No: load dataset in hosted version of google refine (we could run this)
  4. User edits dataset and hits save. As in previous scenario they are prompted to sync the dataset.
1291140609000000 1339774605000000
#895 defect memespring ckan-backlog new Add version number (or simular) to css/js includes query string

Updates to css after a new deploy don't come through without a hard refresh. Adding the version number to the include urls will solve this e.g.

mycssfile.css?v=12345678

1294343382000000 1339774593000000
#909 enhancement pudo ckan-backlog assigned DCat importer for CKAN

Write an importer that supports most well known variants of DCat in importing the data as CKAN packages. In particular, the following sources should be supported:

  • CKANrdf generated exports
  • opengov.se RDF (not really DCat)
  • Sunlight Nationdatacatalog as harvested by dcat-tools.
1295265958000000 1346669602000000
#924 enhancement dread ckan-backlog new Search box has no search button

The search box at the top-right of CKAN's page doesn't have a 'go' button. I feel that a larger percentage of users expect a 'go' or 'search' button on the right-hand side of the box to press to start searching. Techies tend to know the keyboard shortcut of pressing 'carriage-return' but it might be better to follow standard practise on this.

Examples with 'search' button: Internet Explorer, Firefox, Google, Amazon, trac Examples without: ?

1295867533000000 1323170436000000
#925 defect dread ckan-backlog closed fixed Change the search box icon to remove the down arrow

Is there a good reason why the search box has a 'down arrow' icon when there is no drop-down menu? Or can this be usefully removed?

1295867593000000 1323168588000000
#952 task wwaites pudo ckan-backlog closed invalid CKAN should run under nginx/uswgi

second part of #908

1296730498000000 1310125204000000
#979 enhancement kindly dread ckan-backlog assigned Edit Resource extras in the API

Follows on from #826. We can now edit resource extras in the WUI (to some extent - see #978 for remaining issues) and we can view resource extras in the API, but we can't yet edit them in the API.

1297429777000000 1315222244000000
#1004 defect zephod dread ckan-backlog closed fixed Group creation instructions missing

Need instructions on the group page to tell people they need to login to create a group. Someone must have deleted this.

1298559144000000 1323195485000000
#1009 enhancement pudo rgrp ckan-backlog new Improvements to user accounts sytem
  • Forgot password (email a new password)
  • Confirm email
  • Do not show register page if you are logged in (redirect to home page)
  • ticket:1010 - listing of users.
    • Do not use /user for general user account home page (either user normal user page /user/{id} or /user/myaccount)
1298635991000000 1310128574000000
#1041 enhancement thejimmyg thejimmyg ckan-backlog assigned Start Using the CKAN Wiki for Tutorial-style documentation

For example, I will document the following:

I'd love if someone else would write:

  • An authorisation tutorial covering the core model, the command line tools and examples of every possible way of using the system
  • A HOWTO guide with screenshots for adding a package
1300284715000000 1312372367000000
#1062 defect johnglover sebbacon ckan-backlog assigned Data preview encoding error

The preview of "Species Misc Turtle Download" at http://ckan.net/package/taxonconcept results in the following error:

Unable to Preview - Had an error from dataproxy: Data Transformation Error (Data transformation failed. Reason: 'utf8' codec can't decode byte 0x8b in position 1: unexpected code byte

1301396143000000 1311773731000000
#1077 enhancement kindly rgrp ckan-backlog new Move to simpler vdm system

Option 1: 'Changeset' Model

See ticket:1135 for vdm ticket. This would involve a) moving to changeset in vdm b) doing the migration in ckan to support this.

Have developed a new "changeset" based model for revisioning in vdm.

Implementation

  • The main challenge with this change is schema and data migration

Every revisioned object has a revision_id and revision attribute.

Approximate algorithm:

Revision -> Changeset

for revtype in [PackageRevision, ...]:
    for pkgrev in package_revision:
        changeset = lookupchangeset(package_revision)
        ChangeObject(cset, (table, id), dictize(pkgrev))

Question:

  • does pkg include tags attributes or not? or we have to dictize, pkgrev, pkg2tagrev, and tag. Probably the latter.

Option 2: Simplify Revision Object Model

Just use a simpler vdm, see ticket:1136 (move to SessionExtension) and ticket:1137 (remove need for statefulness in vdm).

Discussion

Advantage of Option 1 versus 2:

  • Easier support for pending state and similar behaviour
  • No need to introduce new tables (and hence migrations) when making something revisioned (or not).

Disadvantages

  • Migration is required
  • More difficult to query revision history.
    • Could be addressed by having ChangeObject have separate cols for table name and id but would likely be more difficult.
  • Performance (?)
    • Have one big ChangeObject table to query when looking at changed objects rather than many revision tables.
      • Not sure this is a biggie as even with Revision model biggest revision object tables are probably on the order of the ChangeObject table

Conclusion

Implement Option 2 and leave Option 1 for present.

Option 1 includes Option 2 so it seems that that is required in either case (so we may as well with Option 2).

Option 1 requires significant effort (esp migration) so leave for present and then review the situation at some later date.

1302304464000000 1340034345000000
#1101 enhancement sebbacon ckan-backlog new Integrate googlanalytics into site nav

There's a stats plugin (e.g. at http://trac.ckan.org/ticket/832).

Output from the googleanalytics plugin should append to that page, if the stats plugin is present.

Possibly the stats plugin and the googleanalytics plugin should be merged?

Finally, if the stats plugin is active, then a link to the stats page should be added to the main site footer.

1303393926000000 1339774582000000
#1120 enhancement tsm ckan-backlog new Atom feeds of each tag

Tags could/should have an Atom feed. This would mean that every edit to relevant packages could be easily monitored. See [1].

[1] http://lists.okfn.org/pipermail/ckan-discuss/2011-May/001162.html

1304309285000000 1339774568000000
#1132 defect johnlawrenceaspden ckan-backlog closed invalid test_authz doesn't run

Trying to run the tests in test_authz.py with

$ nosetests --ckan ckan/tests/functional/test_authz.py results in no tests being run:


Ran 0 tests in 0.840s

OK (SKIP=3)

1304966923000000 1307352675000000
#1134 CREP amercader ckan-backlog new CREP0003: Description and Configuration of Harvesters

Proposer: Adrià Mercader

Abstract

The new harvester interface allows to create harvesters for different sources, but right now harvesters don't have many ways to describe and configure themselves. We need a way of allowing them to:

  • Expose their type and other details so they can be used internally and on the UI.
  • Define configuration settings for particular harvester instances.

The Problem

Harvester description

The current UI for adding and editing harvest sources is the same used in ckanext-dgu, and thus the 3 harvester types used in DGU to harvest various GEMINI realted sources are hardcoded in the form. The form will be migrated to a DGU-independent one, so we need the harvesters to provide all the necessary data. There is a current get_type method that returns the harvester type, but for make it compatible with the DGU forms, it returns a machine-readable string (e.g. "CSW Server"), making it error prone.

Arbitrary configuration

In the current implementation, when the harvest process is started, ckanext-harvest looks for all the available plugins that implement the IHarvester interface and calls the appropiate methods for the current stage (gather_stage,fetch_stage,import_stage). At these stages, harvesters have no way of applying arbitrary configuration options, so all harvesters of the same type behave on the same way. For instance, the CKAN harvester needs a way to define the API version to use when harvesting remote instances (Right now, the version 2 is hardcoded on the code).

Specification

Harvester description

Harvesters will need to provide the following information so the UI form can be built:

  • name: machine-readable name (e.g. "waf"). This will be the value stored in the database, and the one used by ckanext-harvest to call the appropiate harvester.
  • title: human-readable name (e.g. "Web Accessible Folder (WAF)"). This will appear in the form's select box.
  • description: a description of what the harvester does (e.g. "A Web Accessible Folder (WAF) displaying a list of GEMINI 2.1 documents"). This will appear on the form as a guidance to the user.

The way to provide it will be an info method that all harvesters must implement, which will return a dictionary with the previous elements:

    {
        'name': 'csw',
        'title': 'CSW Server',
        'description': 'A server that implements OGC's Catalog Service 
                        for the Web (CSW) standard'
    }

Arbitrary configuration

As different harvesters will have very different needs, we need to provide a way to persist arbitrary configuration flags for each harvest source. The more flexible way given the current architecture in my opinion would be to store the configuration options as a JSON encoded object as a property of the harvest source (There already is an unused DB field called config in the database) (Maybe using JsonType??).

This will mean adding an extra field in the harvest source form to allow entering the configuration. This could be just a simple text field where users enter the JSON encoded object or a more clever mechanism (i.e an "Add a configuration flag" link that adds two new text fields for the key and value for each flag, and a mechanism to later build the JSON object). In any case, this should probably be hidden in an "Advance options" section.

Why do it this way

Harvester description

The info method would provide a single point to get all the information related to the harvester, and future properties could be added to the dictionary returned without having to modify the interface.

Arbitrary configuration

There is an already existing config field in the database, so we won't need to change the model. Harvesters could access the config object at any of the stages. Of course they could provide default values in their implementations so users don't need to enter them everytime.

Implementation plan

Deliverables

Risks and mitigations

The highest risk on the harvesters info method side is that harvester implementation don't offer one of the necessary properties (namely name and title). This could fire a warning when showing the UI form or using the CLI.

Participants

Adrià Mercader to do it.

Progress

None yet.

1305108868000000 1339774554000000
#1139 enhancement lucychambers lucychambers ckan-backlog closed fixed Create CKAN Theme Gallery

Take screenshots of existing ckan instances esp those mentioned <http://wiki.ckan.net/Theming> and put on flickr in ckan or ckan-theme group so we can create a gallery ... (both to illustrate theming but also to show ckan instances that are around -- could add to http://wiki.ckan.net/Instances)

1305645859000000 1306941356000000
#1141 CREP johnglover ckan-backlog closed fixed [super] Moderated Edits User Interface

Proposer: John Glover
Seconder: James Gardner

Abstract

We are trying to achieve these goals:

  • To get people involved with making edits to CKAN metadata.
  • To have an ownership model as to who can moderate and validate these changes
  • To not put too huge a burden on these owners.

This feature allows anyone to edit a package and create a new revision, but requires an owner/moderator to approve a revision before it is are made "official".

There have been a lot of discussions around the revisioning system side of this ticket (CREP 0002) and I think these are now largely resolved. We now want to discuss the user interface.

The Problem

We require the following functionality:

  • Allow a group of changes to be stored as a new revision.
  • Allow a linear stack of "community" revisions.
  • Provide a way for the editor and moderator to compare previous revisions to the current one.
  • When a moderator approves a change it creates a new revision flagged "moderated" (this is analogous to a merge commit)
  • Provide a way for the editor and moderator comment on revisions if necessary.

Extra features:

  • Need a way to summarise the changes (as part of the preview perhaps)
  • Sysadmin needs to purge a revision completely

Specification

UI/UX

UI Mockup:

Revisions:

  • Revisions are per package rather than per field.
  • Internally CKAN has separate revisions for resources, extras and package metadata. From a user's point of view this could be confusing to expose, so everything that they see on a package form when they hit save is a single revision.

On the Edit page:

  • We have a panel on the right, listing all the revisions with the current moderated one selected. Moderated revisions are highligted in some way (red and bold?).
  • The values displayed in the form are by default populated from the latest revision (whether community or moderated)
  • Under each field is a "shadow", showing the value of the field in the revision selected in the panel, if it is different from the value in the field. By default the shadow values are populated from the latest moderated revision which is the one selected in the revision panel by default too.
  • When you change the value of a field, a shadow may appear or disappear accordingly. If they disappear a box saying that they are the same replaces it
  • If you want to edit values from a previous revision, you first select that revision to get the shadows populated. There is a button named "Replace fields with values from this revision" under the revision list. You click this, a warning pops up and then you say "Yes". You then select the moderated revision again.
  • We also allow package comments the same way as the todo extension works at the moment. Additionally, we need to be able to differentiate between what the moderator wrote and what a community member wrote, and so we may need to make a small change to the todo extension to facilitate this.
  • In addition to package comments, each revision will have a revision log (analogous to a commit message).

Technical Details

  • This CREP will result in a new CKAN extension.
  • It depends heavily on the new revisioning system (CREP0002), some of the details of which are yet to be finalised.
  • This CREP therefore requires working closely with David Raznick to come up with an API that the UI AJAX calls can use.
  • We will then use suitable test data to mimic these API calls until CREP0002 is ready.

Why do it this way

This hopefully provides a clear and consistent mechanism allowing both a community member to make new revisions and a moderator to view and approve revisions, with largely the same UI/UX.

Implementation plan

Deliverables

A new CKAN extension, consisting of:

  • Code: Python, HTML, CSS, Javascript
  • Unit tests
  • Localization
  • Documentation

Participants

John Glover to do it.

Progress

John has implemented the bulk of this UI. Just some things to tidy up before it is complete:

  • Genshi stream filters to be updated with CKAN 1.5 / 1.5.1 templates
  • history_ajax / read_ajax to be replaced with calls to Action API (or Util REST API)

I've split these two off into a new ticket #1604.

Related Progress

The Todo extension is written and available at: https://bitbucket.org/johnglover/ckanext-todo.

In the section 'The Problem', under extra features, we mention a need for the sysadmin to be able to purge a revision already. This is already done.

See also

#1129 Backend work

1305721003000000 1325352507000000
#1144 enhancement timmcnamara ckan-backlog new Support DSPL

DSPL, the Dataset Publishing Language, is being promoted by Google for its "Google Public Data Explorer" system. It is an XML format with metadata.

The format is described on the developer docs ofthe Google Code site.

Google provides a Python script which reads CSV data and generates DSPL

Sample from http://code.google.com/apis/publicdata/docs/dspl_sample.html:

<?xml version="1.0" encoding="UTF-8"?>
<dspl xmlns="http://schemas.google.com/dspl/2010"
    xmlns:geo="http://www.google.com/publicdata/dataset/google/geo"
    xmlns:geo_usa="http://www.google.com/publicdata/dataset/google/geo/us"
    xmlns:time="http://www.google.com/publicdata/dataset/google/time"
    xmlns:quantity="http://www.google.com/publicdata/dataset/google/quantity"
    xmlns:entity="http://www.google.com/publicdata/dataset/google/entity">

  <import namespace="http://www.google.com/publicdata/dataset/google/time"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/quantity"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/entity"/>
  <import namespace="http://www.google.com/publicdata/dataset/google/geo"/>
  
  <info>
    <name>
      <value>My statistics</value>
    </name>
    <description>
      <value>Some very interesting statistics about countries</value>
    </description>
    <url>
      <value>http://www.stats-bureau.com/mystats/info.html</value>
    </url>
  </info>

  <provider>
    <name>
      <value>Bureau of Statistics</value>
    </name>
    <url>
      <value>http://www.stats-bureau.com</value>
    </url>
  </provider>

  <topics>
    <topic id="geography">
      <info>
        <name><value>Geography</value></name>
      </info>
    </topic>
    <topic id="social_indicators">
      <info>
        <name><value>Social indicators</value></name>
      </info>
      <topic id="population_indicators">
        <info>
          <name><value>Population indicators</value></name>
        </info>
      </topic>
      <topic id="poverty_and_income">
        <info>
          <name><value>Poverty & income</value></name>
        </info>
      </topic>
      <topic id="health">
        <info>
          <name><value>Health</value></name>
        </info>
      </topic>
    </topic>
  </topics>

  <concepts>
    <!-- As noted in the tutorial, this concept should extend quantity:amount.-->
    <concept id="population">
      <info>
        <name>
          <value>Population</value>
        </name>
        <description>
          <value>Size of the resident population.</value>
        </description>
      </info>
      <topic ref="population_indicators"/>
      <type ref="integer"/>
    </concept>

    <!-- This country concept is defined for educational purposes only. A country
    concept exists in the Google geo dataset. See:

    http://code.google.com/apis/publicdata/docs/canonical/geo.html --> 
    <concept id="country" extends="geo:location">
      <info>
        <name>
          <value>Country</value>
        </name>
        <description>
          <value>My list of countries</value>
        </description>
      </info>
      <type ref="string"/>
      <property id="name">
        <info>
          <name><value xml:lang="en">Country name</value></name>
          <description>
            <value xml:lang="en">The official name of the country</value>
          </description>
        </info>
        <type ref="string"/>
      </property>
      <table ref="countries_table"/>
    </concept>

    <!-- This US state concept is defined for educational purposes only. A US state
      concept exists in the Google geo US dataset. See:

      http://code.google.com/apis/publicdata/docs/canonical/geo.us.html --> 
    <concept id="state" extends="geo:location">
      <info>
        <name>
          <value>State</value>
        </name>
        <description>
          <value>US states</value>
        </description>
      </info>
      <type ref="string"/>
      <property concept="country" isParent="true"/>
      <table ref="states_table"/>
    </concept>

    <concept id="gender" extends="entity:entity">
      <info>
          <name>
          <value>Gender</value>
        </name>
        <description>
          <value>Gender, Male or Female</value>
        </description>
        <pluralName><value>Genders</value></pluralName>
        <totalName><value>Both genders</value></totalName>
      </info>
      <type ref="string"/>
      <table ref="genders_table"/>
    </concept>

    <concept id="unemployment_rate" extends="quantity:rate">
      <info>
        <name>
          <value>unemployment rate</value>
        </name>
        <description>
          <value>The percent of the labor force that is unemployed, not seasonally
            adjusted.</value>
        </description>
        <url><value>http://www.bls.gov/cps/cps_htgm.htm</value></url>
      </info>
      <topic ref="social_indicators"/>
      <type ref="float"/>
      <attribute id="is_percentage">
        <type ref="boolean"/>
        <value>true</value>
      </attribute>
    </concept>

  </concepts>

  <slices>
    <slice id="countries_slice">
      <dimension concept="country"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <table ref="countries_slice_table"/>
    </slice>

    <slice id="states_slice">
      <dimension concept="state"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <metric concept="unemployment_rate"/>
      <table ref="states_slice_table"/>
    </slice>

    <slice id="countries_gender_slice">
      <dimension concept="country"/>
      <dimension concept="gender"/>
      <dimension concept="time:year"/>
      <metric concept="population"/>
      <table ref="countries_gender_slice_table"/>
    </slice>

  </slices>

  <tables>
    <table id="countries_table">
      <column id="country" type="string"/>
      <column id="name" type="string"/>
      <column id="latitude" type="float"/>
      <column id="longitude" type="float"/>
      <data>
        <file format="csv" encoding="utf-8">countries.csv</file>
      </data>
    </table>

    <table id="countries_slice_table">
      <column id="country" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
      <data>
        <file format="csv" encoding="utf-8">country_slice.csv</file>
      </data>
    </table>

    <table id="states_table">
      <column id="state" type="string"/>
      <column id="name" type="string"/>
      <column id="country" type="string">
        <value>US</value>
      </column>
      <column id="latitude" type="float"/>
      <column id="longitude" type="float"/>
      <data>
        <file format="csv" encoding="utf-8">states.csv</file>
      </data>
    </table>

    <table id="states_slice_table">
      <column id="state" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
      <column id="unemployment_rate" type="float"/>
      <data>
        <file format="csv" encoding="utf-8">state_slice.csv</file>
      </data>
    </table>

    <table id="genders_table">
      <column id="gender" type="string"/>
      <column id="name" type="string"/>
      <data>
        <file format="csv" encoding="utf-8">genders.csv</file>
      </data>
    </table>

    <table id="countries_gender_slice_table">
      <column id="country" type="string"/>
      <column id="gender" type="string"/>
      <column id="year" type="date" format="yyyy"/>
      <column id="population" type="integer"/>
      <data>
        <file format="csv" encoding="utf-8">gender_country_slice.csv</file>
      </data>
    </table>
  </tables>

</dspl>
1305763609000000 1339774517000000
#1145 enhancement timmcnamara ckan-backlog new Support the Handle System

The Handle System is an initiative to provide persistent references for resources. That is, it's basically a proxy system for preventing link rot.

Its documentation is here: http://www.handle.net/. Servers running CKAN could host a "Local Handle Service", which redirects a hash of a resource to an actual URL.

Some suggested use cases:

  • Researcher would like to cite where data came from
  • Agencies would like to have a way to prevent vendor lock-in from CKAN if they decide to move to another platform
1305764775000000 1339774502000000
#1152 enhancement amercader amercader ckan-backlog new True support for generic CSW servers

The CSW harvesters implemented at the moment were developed with the DGU project in mind, and they assume all remote CSW servers to implement the Gemini 2 specification. Gemini 2 is the profile defined in the UK for INSPIRE complying metadata, so obviously catalogs from other countries or non-INSPIRE complying ones won't be able to be harvested.

The changes needed to support generic CSW servers (i.e. those implementing the ISO 19139 profile) are:

  • Handling the validators (right now are hardcoded in the harvester

code). This probably involves issues discussed in the CREP 3 (ticket #1134)

  • Changes in the model to adapt the specification to ISO 19139
  • Renaming objects and classes which are now Gemini-centric

List of CSW servers tested:

https://spreadsheets.google.com/spreadsheet/ccc?key=0Atp3cZFjuIOAdDBVQWRINnlfN1d0b2lleHVEdjBSb2c&hl=en_US&authkey=CNu4hsEB#gid=0

1306141334000000 1313411822000000
#1153 enhancement lucychambers lucychambers ckan-backlog closed fixed Update CKAN wiki front page

Update CKAN wiki front page - a la OpenSpending?: http://wiki.openspending.org/Main_Page

Sections should relate to different types of people using the site:

Developers, Users etc..

1306155211000000 1306941386000000
#1163 enhancement rgrp rgrp ckan-backlog new Improvements to Storage Extension

Storage is now working but there are

  • Integrate with Resources (e.g. create a resource for each file upload and give option to associate with a package)
    • Should we introduce rule that files *not* associated with a Resource are periodically deleted?
  • Allow setting of a file name/path before upload
  • Allow for file overwriting/deleting etc (how should this work -- do we want to allow this sort of thing)
  • Integrate local file upload stuff in api/auth/*

Different Backend Issues

Local file store is rather different from 'remote' storage in various ways:

  • For remote you don't want to use many buckets as there are bucket limits while for local you want to. Should we there have a single path that users provide which we then partition differently for different backends.
1306408778000000 1310133808000000
#1168 enhancement thejimmyg dread ckan-backlog assigned Test system for deb packaging

Get buildbot to:

  • build the deb packages
  • install them into a fresh virtual machine
  • run smoke tests on the installed ckan
1306441994000000 1330990423000000
#1179 enhancement timmcnamara ckan-backlog new Support tag aliases

A small number of tags are near-duplicates of each other.

Perhaps we could support word stemming from NLTK and/or manual tag aliases:

statistics statistik ... survey surveying surveys

1307429221000000 1339774332000000
#1182 defect timmcnamara ckan-backlog new Comments from deleted packages appear in "Recent Comments" feed

When a package has been deleted, say for spam moderation, comments still appear in the recent comments section.

This is a problem because non-admin users will be shown a warning that they're not authorised to view the package if they click on the link.

At CKAN.net currently, this affects the most recent comment.

1307658251000000 1339774319000000
#1184 enhancement timmcnamara ckan-backlog new Support Wuala as CKAN storage option

Most of CKANs storage options are tied to the USA. This brings concerns of data security for some organisations who may wish to adopt the system. Wuala is a distributed file system that stores data in a peer-to-peer manner. The company behind it, LaCie? sells storage for a fee. However, they also enable clients to have 'free' storage space when machines act as a storage node.

In order to be a storage node, a machine needs to be online for more than 14% of the time - roughly 4h per day. Most CKAN servers are likely to have a far greater uptime than this.

Supporting Wuala would go some way to enabling CKAN to be used in a secure manner. That is, CKAN could be promoted for organisational use where there is lots of data to be stored and large geographic distances to be managed. There is a Python client available and a fairly long Google Tech Talk that overviews the system.

1308034751000000 1339774306000000
#1185 defect timmcnamara ckan-backlog new Administrators can't delete packages from web UI

Administrators have "View", "Edit" and "History" tabs. However, I can't see a way to delete a package from the web UI.

Version: CKAN.net as of today

1308111818000000 1339774289000000
#1188 enhancement nickstenning ckan-backlog new Allow diffing against initial (blank) package version

Currently the history page only allows diffing between different versions of a package, but there doesn't appear to be any easy way to see the changes introduced by the first version of a package.

I'm requesting the ability to diff against a "blank slate" initial state of a project, so I can see the content of the first project commit.

Not sure if this is a vdm feature, so I'm putting this ticket in against ckan.

1308153160000000 1339774275000000
#1197 enhancement timmcnamara ckan-backlog closed wontfix Add JavaScript guide for CKAN

A new library, guiders.js, has been open sourced that seems to be great at unobtrusively introducing new users to features of websites.

The library drives Optimizely's website. The GitHub repo is here: https://github.com/jeff-optimizely/Guiders-JS

Some examples:

http://posterous.com/getfile/files.posterous.com/temp-2011-06-22/BnlgHDvhjpFfqbwvprsIaklqrpdgJyebfuhxajbDhmvnFgzdzzCJdvluJFki/Screen_shot_2011-06-22_at_6.42.21_PM.png.scaled500.png

http://s3.amazonaws.com/files.posterous.com/temp-2011-06-22/CpvlcinkrxhtnAHfGtEHhxyGkraqlFmCjcnztFuugagBGxmBjCdwuquHpvEB/promo_gmail.png

1308801032000000 1340020357000000
#1198 enhancement dread ckan-backlog new Publisher hierarchy

'Publisher' entities in the model. They are hierarchical.

'User-Publisher' connections with one or more roles (e.g. drafter, moderator).

Authorization settings can control who can set what values in a 'published by' type field.

Publishers and User-Publishers available to read in the API.

Future tickets will provide:

  • API to write Publishers and User-Publishers
  • UI to edit Publishers and User-Publishers

(This feature deprecates authorization groups)

1308820592000000 1339774200000000
#1201 enhancement kindly ckan-backlog new seperate out logic in atom feeds to logic layer.

Simplify the logic in the atom feed an make all feeds use logic layer to return lists.

1308928892000000 1310124297000000
#1203 defect johnglover rolf ckan-backlog new Moderated edits: html code shows as "changed" although it is not

I've installed the Moderated Edits extension (ckanext-moderatededits) and am editing a package imported from IATIregistry.org, with an extra field which contains a bit of HTML.

The editor indicates the field has changed, although the content hasn't (see screenshot). All I can find so far is a minor difference: in the field content, there is a code &#8212 and in the rendered table that is an &mdash;

1309274970000000 1313401579000000
#1227 enhancement timmcnamara ckan-backlog new Display packages' tags in search results

In when displaying search results, it would be useful to also display the tags of a package. Sometimes it's difficult to infer the scope of what the package does from the title and the first sentence of the description. Tags are quite concise way to display rich information.

ENV=datacatalos.org, with CKAN 1.4.2a

1311034262000000 1339774147000000
#1231 requirement kindly thejimmyg ckan-backlog closed wontfix [super] Management Information Reporting

Child tickets:

  • #1101 Integrate stats and googlanalytics into site nav

We have a spreadsheet from UKLP of statistics we'd like to generate

1311173919000000 1325474447000000
#1232 enhancement thejimmyg ckan-backlog new [super] Interface improvements

Child tickets:

  • #1194 "Welcome back" message for newly registered user
  • #1202 Links to datapkg utility don't lead to info about it
  • #925 Change the search box icon to remove the down arrow
  • #923 Search box doesn't work in leaderboard page in stats extension
  • #1034 Flash message cached
  • #737 Markdown syntax summary page
  • #811 Extra field editing form layout breaks when there are long field names
1311178296000000 1315948536000000
#1233 enhancement thejimmyg ckan-backlog new [super] Improve wiki-style functionality for history

At the moment we have a good revisioning system but a poor history interface. We need to improve this in a number of areas:

  • #191 Searching by modification date
  • #193 Searching by time-related field
  • #301 Package discussion pages
  • #1236 Package history page should provide links to pages at particular revisions, similar to the wikipedia pages
  • #1236 Viewing old revisions or unmoderated changes should have a message at the top of the package page
  • Other improvements as per my word doc.
1311179392000000 1315948668000000
#1240 enhancement kindly rgrp ckan-backlog assigned [super] API v4

(Just creating this ticket as somewhere to keep notes)

  • Decide on REST api versus action API
    • Do we want to support both?
  • Tidying
    • Unify on /api/v{version num}/... structure (do we want a default option that points to current default? e.g. /api/default/ ...)
    • extras merged into normal field list in package
    • Get rid of /rest/ so just have api/v1/package
    • Get rid of separation of search api from 'rest' api
      • Propose that GET on REST index is search e.g. /package/?q=...
        • This is also resolves issue whereby GET at root returns whole package set (a *bad* idea) as this would now become the matchall search query (with a default limit on items returned)
  • Resource read/write in API (separate from package)
    • Does this need authorization work?
  • user/account API - read/write
  • Remove autocomplete -- can just use search
    • Do not worry about backwards compat as should only be used in our js (if others using it too bad!)
1311525660000000 1325473312000000
#1251 enhancement rgrp rgrp ckan-backlog closed fixed Rename wiki.ckan.net to wiki.ckan.org

Inline with general naming policy going forward.

1311965147000000 1317315159000000
#1255 enhancement kindly kindly ckan-backlog assigned Drupal consistancy checks.

Make a robust way to make sure the drupal database is consistent with the ckan data.

1312219968000000 1313400054000000
#1257 enhancement dread ckan-backlog new Anti-Spam tools

We are getting more and more spam on ckan.net and we need to improve our strategy of combating it. It is bad because google ranks who we link to (which we do want for legitimate links), and our front page contains the latest package edits, so spam is immediately very visible.

Spam:

  • Package creation
  • Package edit
  • User creation
  • Group creation

Systems to consider:

  • Automatic
    • captcha
    • bayesian scoring
  • Sysadmin
    • a tool to quickly analyse and remove packages, edits and users
  • Helpful users
    • button to click indicating spam, leading to any/all of:
      • add TODO item (which emails package admin and sysadmins)
      • quarantine the package/user so it doesn't show up on the front page and a warning is shown when you view it

General thoughts:

  • We should learn from the wikipedia tools, as they are the experts
  • This is a serious problem that is only going to get worse and is a threat to the whole site.
1312283565000000 1339774129000000
#1259 enhancement johnglover pudo ckan-backlog new "Add a row" for Extras on Package form

The default package form offers 4 empty extras fields. Like the resource section, it should have an "add more" button to add another row.

1312302693000000 1312907056000000
#1260 enhancement pudo ckan-backlog new Remove duplicate functions from _util.html

There seems to be both a list view for dictized and non dictized data structures for all entities in _util.html at the moment. Probably in the back of someone's mind already, but cleanup here would be nice.

1312366652000000 1313401499000000
#1261 defect pudo ckan-backlog new Investigate dots in extras search

It seems that searching for extras_foo:value works with solr, but extras_foo.bar:value doesn't. No theory on why.

1312366768000000 1312366768000000
#1262 enhancement pudo ckan-backlog new Enforce "create-user" permission

This does not seem to have any implications at the moment, it should lock down registration and remove all related links.

1312375296000000 1323090112000000
#1269 defect johnglover rgrp ckan-backlog closed wontfix User view page shows the packages followed by viewing user not user being viewed

This overlaps with Sean's follow support (for activity streams) and so will be dealt with there.

1312906598000000 1335875028000000
#1273 requirement amercader ckan-backlog new Create docs for API v3 1313412083000000 1313412083000000
#1278 enhancement amercader ckan-backlog new Refactor authorized_query calls

There are some functions that still use the Auhtorizer().authorized_query method:

./ckan/controllers/authorization_group.py:24:        query = ckan.authz.Authorizer().authorized_query(c.user, model.AuthorizationGroup)
./ckan/lib/base.py:237:        groups = ckan.authz.Authorizer.authorized_query(c.user, model.Group, 
./ckan/lib/search/sql.py:55:        q = authz.Authorizer().authorized_query(username, model.Group)
./ckan/lib/search/sql.py:118:        q = authz.Authorizer().authorized_query(self.options.get('username'), model.Package)
./ckan/logic/action/get.py:154:    query = Authorizer().authorized_query(user, model.Group, model.Action.EDIT)

./ckan/tests/test_authz.py:158:        q = self.authorizer.authorized_query(self.notadmin.name, model.Package)
./ckan/tests/test_authz.py:353:        q = self.authorizer.authorized_query(self.notmember.name, model.Package)
./ckan/tests/test_authz.py:357:        q = self.authorizer.authorized_query(self.member.name, model.Package)
./ckan/tests/functional/test_authorization_group.py:44:        group_count = Authorizer.authorized_query(u'russianfan', model.AuthorizationGroup).count()
1313415177000000 1313415177000000
#1280 defect kindly ckan-backlog closed fixed fix sqlalchemy so that it works with postgres 9.0

After the ckan migrate process, sqlalcehmy reflect does not work and causes an error when reflecting indexes. A bug report needs to be put into sqlalchemy.

1313513090000000 1328786670000000
#1283 enhancement dread dread ckan-backlog closed fixed Deleted packages shouldn't be searchable or browsable

A package in deleted state doesn't show to general users, but admins DO see them. The idea is that they can resurrect them if they want. But now we have the trash can that lists deleted packages. So we don't want admins to see the deleted packages anywhere else.

(But an admin should still be able to READ and EDIT a deleted package, so if he clicks on it in the trash list he can see it to decide whether to resurrect and actually be able to change its state.)

Same in the Web interface and API.

This ticket ties in with #948 highlighting a package being deleted (when viewing or editing) and follows on from the introduction of the trash can #1076.

1314112481000000 1330083933000000
#1286 enhancement rgrp ckan-backlog new Remove remaining formalchemy stuff

Stuff I've spotted:

  • forms/*
  • template/group/edit_form.html
  • template/package/edit_form.html

This can go once new DGU form is in.

1314116996000000 1342436420000000
#1287 enhancement thejimmyg dread ckan-backlog assigned NAVL validation errors - Junk fields should be listed explicitly

When you create a package, but specify a key that is not allowed (e.g. 'relationships') then you get error message:

{"__junk": ["The input field __junk was not expected."]}

It should mention the actual key which is not expected. e.g.

{"relationships": ["The input field 'relationships' was not expected."]}

Kindly said that James' version of NAVL was better in this respect, so this might be best solved by moving to that.

1314203102000000 1330990459000000
#1288 defect dread ckan-backlog new Package edit/creation can't include 'relationships' field

When you create or edit a package (via the API), you aren't able to specify the relationships it has. (If you do you get 409 {"__junk": ["The input field __junk was not expected."]} )

The normal way to create relationships is via /api/rest/relationships/ and this works. But when you GET a package, the dictionary lists all relationship details. So this bug creates a problem for editing a package that has relationships - you want to GET it, make any edits and then PUT it back. The work-around is to delete the 'relationships' key from the dict before you PUT it back.

Options

Ideally, CKAN would read the 'relationships' key and make the necessary changes. This is a chunk of work.

Another good option is to allow an unchanged 'relationships' value, but barf it is edited. This is also a chunk of work.

A bad option would be to just ignore the 'relationships' value, since users will get frustrated changing this value and wonder why it never saves, not understanding it is different to all the rest, without error message.

A final option is to get rid of relationships altogether.

1314203695000000 1339774098000000
#1289 CREP dread ckan-backlog closed wontfix Remove 'relationships'

Abstract

Package Relationships have not taken off in the 18 months we've had them in the API. There are some issues with them and we need to spend more time improving them or consider getting rid of them.

The Problem

Original use cases are expressed here: #253 Here are comments about how we could handle these specific examples better:

  1. groups of packages - maybe better with a custom tag?
  2. fragment resources - soon to be covered by 'kind' resource field #957

3&5. derived resource - better to have some sort of resource relationship perhaps?

  1. linked resource - again better to have some sort of resource relationship perhaps?

Outstanding issues needing serious effort to fix:

  • #256 Editing them in Web UI (not done yet)
  • #1288 Package edit/creation can't include 'relationships' field

Specification

Remove relationships from model, API, tests, Web UI. Data migration to remove from db.

Why do it this way

Getting frustrated having problems with the code, when it's not used much. Often asked about what it's for, but rarely used. Seems an overly complicated design.

Backwards Compatibility

n/a

Implementation plan

Deliverables

See Specification

Risks and mitigations

Risk: a customer suddenly wants this, and the new ways to relate resources are not in place yet.

Mitigation: discuss this decision thoroughly to make sure we are confident the use cases are not important. Discuss with team, ckan-discuss and specifically the LOD people who have some related packages on thedatahub.org.

Participants

David Read

Progress

Not yet.

1314206502000000 1317315211000000
#1290 enhancement dread dread ckan-backlog closed fixed Better error when blank database

When installing CKAN, when doing "paster serve development.ini", lots of users encounter the error for every request:

ProgrammingError: (ProgrammingError) relation "user" does not exist

This is because the database tables have not been created - they have forgotten or missed the "paster db init" step.

Can we provide a better error to say that the database is not initialised yet?

Implementation options

  1. At the start of every request we reflect the database tables and check they are there. This is rather expensive!
  1. Uncached requests to the home page start with a cheap database query. If there is an exception then return this error about database setup. I really like Drupal's page for this that has in large letters that the site is currently off-line. Below a line, in small letters, there are developer suggestions on what is wrong and where to look to fix it.
1314264255000000 1314270894000000
#1305 defect nils.toedtmann amercader ckan-backlog closed fixed SMTP config for thedatahub.org and IATI

The email sending functionality (e.g for password reset) does not work on thedatahub.org and IATI (and probably some other instances) when using an address which is not a okfn.org one.

Could not send reset link: SMTPRecipientsRefused({u'amercadero@…': (550, 'relay not permitted')},)

As I said, adria.mercader@… works fine. The SMTP server used mail.okfn.org

1314956657000000 1315317033000000
#1306 enhancement kindly rgrp ckan-backlog closed wontfix Tests for the logic layer

AFAICT there are no tests for the logic layer at the moment. I imagine this is an issue (if it is not please explain and close as wontfix).

1315153267000000 1319797204000000
#1311 enhancement rgrp rgrp ckan-backlog new Modal user register and login form

Subticket of: #1294

Rather than having to visit a dedicated page it would be good if registration and login could be done from a modal form (separate or combined ...).

Why

  • It could be used from dataset creation page in situations where user needs to be registered / logged in to create a dataset (so we could allow someone to start creating a dataset and only get them to login at the end ...)
  • It allows for quicker and easier logging in

Implementation

  • See Friedrich's work on the datahub
1315297227000000 1315297227000000
#1314 enhancement dread ckan-backlog assigned ckanclient search - generator improvements

Apparently the search generator always makes two requests, even if you don't want to see the search results, which might be slow. Can this be optimised?

Maybe we should also provide a second search function that doesn't use the generator - the original simple search function (that leaves the user to deal with limit & offset).

1315395410000000 1340191233000000
#1317 defect dread ckan-backlog assigned password reset - improve user search

In password reset, it gets confused if you have two similar users. This is because with the string the user provides, it searches several fields, not just name but also fullname and email address, allowing you to search for these. But only name is unique.

Specific problem: Ira searches for "Irina" then it finds both: <User name=irina fullname=Irina [email protected] ] and <User name=shevski fullname=Ira email=> (I think)

Maybe need to choose which field it searches?

1315415539000000 1340191221000000
#1326 enhancement thejimmyg ckan-backlog new Write a set of auth plugin functions to integrate with Druapl

Ticket #787 described join auth between CKAN and Drupal. The authentication part is live and implemented. This ticket is a placeholder for work that will be needed in the new auth system to link authorization functions to Drupal. It is dependent on the groups refactor.

1315821084000000 1315821084000000
#1330 enhancement rgrp ckan-backlog closed invalid Deprecate / Remove test_authz.py

test_authz.py appears to test in great detail some very specific additional authz (related to total site lock-down it seems -- introduced I think for hri project).

I think there are simpler ways to get total site lockdown (use external auth!) and this test is slow and delicate (e.g. depends on specific words in templates). Suggest removing. If we don't remove we should at least refactor tests for access to certain pages to use a proper method of testing (e.g. agreed html comments in each page) rather than being depending on the presence of absence of specific wording.

1315899129000000 1327060201000000
#1336 defect johnglover dread ckan-backlog new License fudge

cset:4b59ab34137d ckan/logic/action/get.py:

-            isopen = model.Package.get_license_register()[license_id].isopen()
-            result_dict['isopen'] = isopen
+            try:
+                isopen = model.Package.get_license_register()[license_id].isopen()
+                result_dict['isopen'] = isopen
+            except KeyError:
+                # TODO: create a log message this error?
+                result_dict['isopen'] = False 

This change hides problems with the license server and returns potentially incorrect values for openness.

This has been noted as 'temporary fix' but seems to be forgotten about, since it has been merged to default and gone into release 1.4.3.

I suggest the licenses are cached (I thought this was already the case when CKAN first requests them after start-up?). I suggest failure would return 503.

1315912057000000 1323173073000000
#1339 enhancement kindly rgrp ckan-backlog closed fixed Issues / question re navl and data conversion

I ran into a bug with the size field on resources.

  • It would not accept an empty value from form (IMO this clearly equates to null/None)
  • This could be fixed via using ignore_empty instead of ignore_missing
  • However using this means there was no way to empty the field (e.g. i may just want to set the size field back to null not just change to another value)
  • similar issues could arise around other fields (such as last_modified ...)
    • cf cset:645031d07b60

To solve this (cset:58acdcfe6d4e) i created an int_converter temporarily in logic/schema.py (this is almost certainly the wrong place). But I think it raises a bigger issue about the conversion layer and how it works.

1315947022000000 1316014954000000
#1341 enhancement kindly kindly ckan-backlog reopened Delete spam users from ckan

Spam users where added to thedatahub and we need to clean them.

1315995034000000 1320141540000000
#1343 enhancement rgrp rgrp ckan-backlog new [super] User related improvements (login, user pages etc)
  • Disallow account creation via openid - #1386
  • Require email field - #1319
    • Require email confirmation to be activated (?)
  • Improvements to user page (e.g. show activity and more info about user) - #1396
  • Modal user login - #1311
1316017098000000 1318528138000000
#1352 enhancement amercader ckan-backlog new Use logic functions instead of as_dict when indexing entities

The current search implementation uses the output of the the as_dict method of the domain Package object to update the index

https://bitbucket.org/okfn/ckan/src/56c79e3fc44c/ckan/lib/search/index.py#cl-48

It also uses package_to_api1 in the SynchronousSearch? plugin:

https://bitbucket.org/okfn/ckan/src/f9dfb0506594/ckan/lib/search/__init__.py#cl-93

This prevents extensions from being able to index custom properties (e.g. faceting by custom extras not included in the model).

The search should use the logic function to get the package properties:

get_action('package_show')(context,data_dict)
1316615397000000 1339774086000000
#1355 defect amercader ckan-backlog new Package extras property does not include the newly created ones

The extras in the package object sent to the extensions after editing (https://bitbucket.org/okfn/ckan/src/01efd5649c10/ckan/logic/action/update.py#cl-226) do not include the newly added.

1317034126000000 1339774056000000
#1358 enhancement zephod rgrp ckan-backlog assigned Generate configuration documentation automatically from the deployment_ini_tmpl file

At the moment documentation of config options is duplicated between source (deployment_ini_tmpl in ckan/config which is used to generate user ini file) and the docs.

Suggest we write a script that automatedly generates reference documentation for the config from the source.

May be obsoleted by #277 (some config in db)

1317076350000000 1318257823000000
#1382 defect thejimmyg thejimmyg ckan-backlog new Deleted resources are present for harvested package

Perhaps the importer deletes them before re-importing. We shouldn't have deleted resources, so let's investigate.

1318337889000000 1318337889000000
#1384 task rgrp shevski ckan-backlog new CKAN wiki needs updating to refer to thedatahub.org instead of ckan.net and datasets instead of packages

Most articles still refer and link to ckan.net, wiki.ckan.net and to packages

1318414077000000 1318414077000000
#1388 defect dread ckan-backlog closed fixed etags caching on home page

Needs to update on:

  1. language - It's broken on thedatahub.org if you change to German and then click on the CKAN icon top-left to go to thedatahub.org again.

AND

  1. whether signed in or not - #1373

BUT not latest revisions (which is what it was)

Or get rid of etag caching on this page and others?

1318429443000000 1320174229000000
#1393 enhancement johnglover dread ckan-backlog assigned Don't skip search tests

Now we don't use postgres search, all the tests involving search now don't need to be skipped when running on sqlite. Should help coders spot earlier if these tests break.

1318505453000000 1320153590000000
#1403 enhancement zephod zephod ckan-backlog new Refactor groups index page

Groups are listed alphabetically with paging - not an ideal user experience. We would like to list groups in order of 'popularity': The number of datasets they contain.

Following this chain of thought, then, it would be nice to rearrange the groups table by clicking on column headers and having it sort by that column.

Furthermore, then, we'd like to implement a full-fledged groups search feature (if this is at all feasible).

The forthcoming groups refactor will probably have some bearing on this task.

1318847512000000 1318847566000000
#1406 enhancement zephod ckan-backlog new Re-enable RSS subscriptions

RSS 'subscribe' buttons appeared in many places on the site but were not very helpful. They took (confused) users pointed to the raw feed code, and Google Reader could not understand the feed. Safari, however, could interpret it correctly.

Their presentation needs to be clear and consistent; the RSS feed really needs testing in a variety of readers; and we need to decide exactly which items should get a feed. (Package updates? Groups?)

1318861327000000 1320930088000000
#1411 enhancement zephod rgrp ckan-backlog new Force resource format to be lower case (also mimetype)

Format should be lowercase. Automatically lower case (for extra points have a bit of javascript to force lower case when entering).

  • Even more points: do a update on thedatahub repo to make all format lower case (or script this as an update?)
1319319604000000 1319319604000000
#1413 enhancement seanh shevski ckan-backlog closed fixed Ask users to 'add email address' when logged in

I'd like us to display a one-off banner that prompts users who log into thedatahub.org to go and update & their profile & fill in their email address (if we could do it just for those who don't have an email address, then even better) That is, once they log in, they see a banner which says: Please update your profile [here] and add your email address, so you can receive notifications, be able to reset your password and get updates

Then when we build notifications or if we decide to survey people, we can actually contact them. Email address should be required as standard

DR: Also, mention their Full Name too - v. useful for selecting correct user in group curation

1319453108000000 1330020486000000
#1414 enhancement shevski ckan-backlog new track user log-ins on thedatahub.org

Set up tracking for user logins so that we have stats about how many active users of thedatahub exist want to be able to see who logged in the the last x months

1319454782000000 1319454782000000
#1423 enhancement markbrough ckan-backlog new Edit resources suggestions
  • Description vs Name - Edit Resources view is showing the name of the package rather than the description, and a lot (all?) of the packages before the upgrade don't have names, so might be good to swap this round again, e.g.: http://thedatahub.org/dataset/edit/iati-registry
  • Moving resources - Moving them up or down the list used to be quite useful if you had a lot of resources that you might want to leave on the resources page, but only one or two that were actually current and that you wanted to draw attention to. This doesn't exist any more on CKAN but I think it would be good to add it back in.
1319641906000000 1338203678000000
#1424 enhancement dread ckan-backlog new Openness notice should be clearer

ckan-discuss discussion suggests changes to the 'openness' indicator ( http://lists.okfn.org/pipermail/ckan-discuss/2011-October/001786.html )

Dataset view page:

  • If there is an explicit but non-OKD compliant license, such as CC-BY-NC, then this should be stated explicitly, perhaps: “This dataset is Not Open. License: Creative Commons Attribution Noncommerical. This is not an open license as it does not meet the Open Knowledge Definition.”
  • If the license is marked as “Other::License Not Specified”, then this should be stated explicitly, perhaps: “This dataset is Not Open. It is published without an explicit license, the publisher reserves all rights to the dataset.”
  • 3. If the license field was left empty by the contributor of the Data Hub record, then again this should be stated explicitly, perhaps: “This dataset is Not Open. The license of this dataset is unknown or unspecified. Start an enquiry on IsItOpenData? »
  • There is a bug so that non-open licenses doesn't have an openness notice.
  • If downloadable resources are not available, this should not affect 'openness' - check this has been removed.
1319648089000000 1319648089000000
#1429 enhancement rgrp rgrp ckan-backlog new Provide DOIs for datasets in a CKAN instance

DOI = digital object identifier = http://www.doi.org/

As a Publisher I want a DOI for my dataset so that it can be cited by and linked to by others in a standard and easy way.

Details

  • Probably implement as extension rather than core
1319977305000000 1319977305000000
#1432 enhancement rgrp ckan-backlog new [super] Data processing system for CKAN and Webstore

Super ticket: #1190

A data processing system which utilizes the Webstore. One could get a long way with simple javascript running in the browser for development with this javascript then run offline using something like nodejs. Alternatively one could allow one to specify a url to e.g. a python file which would then be run in a sandbox (with access to some specified set of python modules)

1320142747000000 1339774041000000
#1439 enhancement dread ckan-backlog new Action API discoverablility

A good service API needs to be discoverable, so you are not always having to refer to the documentation html.

Maybe /api/action should return a list of actions available? (Currently this returns a 404.)

  • It would be nice to sort these into get/create/update/delete.
  • #1438 Parameters for each of the actions must be discoverable too

/api/action/{action_name} should also return the help text / parameters allowable. (Currently this returns 400 error)

1320161970000000 1325474974000000
#1447 defect kindly dread ckan-backlog assigned disk space leakage

Periodically we see some CKAN servers fall over because they run out of disk space. We need to find out if there is a common cause and fix it.

One problem in the past has been file handles running out when creating lots of tiny files in the data directory.

Another problem has been several enourmous backups being created every day - pdeu on eu25.

1320666843000000 1340727330000000
#1459 enhancement rgrp rgrp ckan-backlog new Featured Dataset feature

Provide way to mark a dataset as featured. Featured database show up on the front page.

TODO: detail this more.

1321113012000000 1321113012000000
#1466 enhancement thejimmyg ckan-backlog new Need to support https login for multiple instances as part of the CKAN package install 1321375978000000 1328529062000000
#1489 enhancement dread ckan-backlog assigned Updating example theme/extension

ckanext-example needs updating for CKAN 1.5:

  • theme changes
  • new forms

About: 'ckanext-exampletheme' was created in Spring 2011 as an example CKAN extension that showed how to customise the look & operation of CKAN. This moved to github and renamed 'ckanext-example'.

1322137920000000 1324292384000000
#1507 enhancement zephod rgrp ckan-backlog assigned Minor fixes to dataset add on Group edit form - 0.5d

Group edit dataset add form needs some work

  • Dataset name is not cleared when you add
  • No way to remove item from list of datasets to be added if I make a mistake
  • (2nd Apr 2012) It now seems that option to add multiple datasets at once has disappeared (perhaps during the CSS/HTML refactor ...)
1323088429000000 1338205220000000
#1528 enhancement zephod zephod ckan-backlog closed fixed Gravatar next to My Account link on all pages

Rather than the anonymous 'user' icon in the top-right corner, we could display your gravatar on all pages when you are logged in.

This was suggested by Richard Cyganiak: http://lists.okfn.org/pipermail/ckan-discuss/2011-November/001825.html

1323197931000000 1330020444000000
#1533 defect rgrp amercader ckan-backlog closed wontfix Make sure ckanext-wordpresser works on CKAN 1.5.1

Superceded by #2234

Super ticket: #1508

Etherpad: http://ckan.okfnpad.org/wordpresser

The wordpresser extension does not work with ckan 1.5.1. The wordpress page is not loaded.

Looks like this changeset may be involved https://github.com/okfn/ckanext-wordpresser/commit/0efa0649896a7339865f178a302be639ddc561e1

I think that even before that changeset, the page was shown without any margins.

In any case, the extension seems to be too sensible to changes in the ckan layout so perhaps it should take into account the CKAN version and have a fixed set of xpaths and templates that work for each version

1323277660000000 1338202676000000
#1534 enhancement rgrp ckan-backlog new Change revisions to record userid rather than username

The use of username is problematic because username's can change.

  • Change all revision creation code to use user id (simplest is to change c.author field in lib/base.py (?))
    • (?) Add a field ipaddr for ip address of anonymous users? (or just keep putting this in author field on Revision and then acception that those won't match when we do a look up against user table)
  • Change user view page to look up against user id rather than name
  • Perform migration on existing Revision objects
    • Match should probably be against both openid and username when searching Revisions' author field (especially true on CKAN where some people have already changed their username from being their openid)
1323278790000000 1338205050000000
#1535 enhancement dread ckan-backlog new Plump for auth header of: X-CKAN-API-KEY

When using the API, the apikey needs to be supplied in a header called 'Authorization'. Because some proxys / deployments use this header for other things, a configurable header was provided as an alternative, with default "X-CKAN-API-KEY".

Rufus suggests having *one* way for this. a) making this not configurable any more b) making X-CKAN-API-KEY the default

(keep Authorization allowed, but not documented, for backwards compatibility)

1323279082000000 1339774019000000
#1542 enhancement dread ckan-backlog new Buttons to purge spam datasets and groups

A sysadmin should be able to easily examine a suspect group or package, determine if it was created by a spammer (as opposed to being a legitimate object that has been graffitied by a spammer) and purge it.

The existing two-stage revision delete is currently unreliable and perhaps too laborious.

Olav and Richard have needs along this line.

1323364930000000 1339774000000000
#1544 task dread ckan-backlog new delete old git branches

We have 150 odd branches (git branch -a) - most of them old - we should prune them. At very least, branches that have been merged in should be deleted. Look at old branches that haven't been merged in and wonder why.

May be of some use:

git branch --merged
git branch --no-merged
1323702610000000 1323702610000000
#1549 enhancement ross ckan-backlog closed wontfix [super] Short link tool

It would be great to have a CKAN extension that allowed users (or CKAN itself) to generate short links to other URIs (both internal and external). Once created, shortlinks made by CKAN should be changeable. This would allow uploaded content to be moved without the user's link changing at all. The tool itself might also be of use as a general link-shortener to users other than the CKAN system itself.

Another useful feature would be for this to also collect some simple analytics such as the referrer and client IP for future reference. I'm not yet sure what we would do with the analytics other than some sort of popularity metric.

Questions:

  • Core, or Extension, or Self-hosted?
1324036998000000 1325474219000000
#1550 enhancement ross ckan-backlog assigned Allow simple auth via the API

It should be possible to pass userid/username and api key and obtain a response from CKAN for external services that use CKAN auth. Those services shouldn't be talking to the DB directly.

1324049369000000 1346670055000000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Note: See TracReports for help on using and creating reports.