{22} Trac tickets (2647 matches)

• Uses API to change licenses of all packages in a CKAN instance to a particular one.

• Run it on DGU.

e.g. national_child_measurement_programme had no department but "Information Centre for Health and Social Care" for agency, and TNA inferred it should be under department DoH. But now the next ONS update to this series expects a blank department, so creates a new package national_child_measurement_programme_.

Solutions: ResourceSeriesLoader? allows synonyms for values.

CKAN uses a database. We can put the database on another machine and then clone the CKAN machine. Then there are two machines calling the database and converting SQL results into HTTP responses.

How many machines can you have before the database becomes the bottleneck? At that point, what QoS can be obtained with normal hardware?

Afterwards, can CKAN be made to scale further than this with a little development? Can its Postgres database be mirrored in realtime? Or can we change SQL writes (create/updates) to write to many databases?

Are there any other options for scaling the persistence mechanism?

Move the solr SearchBackend? to a seperate code base, remove any mentions of the CKAN model.

Add a config option 'template_footer_end' which is inserted in layout_template just before </body>.

This allows sysadmins to add extra items, especially, scripts directly into site without having to do any theming and is especially useful for things like google analytics.

Aside: going forward may want to turn this into a extension.

Cost: 1h

• Add help text, drawing from:
  • ca.ckan.net
  • ​http://wiki.okfn.org/ckan/doc/package
  • ​http://wiki.okfn.org/ckan/doc/faq
• Make longer text revealed on click

To do this for the UKLP, the Ordnance Survey would need to add to their map service an operation to return image for a given bounding box, and COI would need to present the image on the search result page.

Otherwise, it would be possible to use Google.

Where said "v0.4" would be the result of converging four lists mentioned in requirement #711.

Needs to delete all harving jobs and harvested documents, and probably to purge all packages (perhaps just delete?).

The search field (on the homepage and in the top right corner of each page) should have autocomplete for package name. If a package name is selected, it should not do a search but go straight to the package page.

A link “About CKAN” pointing to ​http://ckan.net/about should be visible on every page. This is important because people often land on subpages and may have trouble figuring out what CKAN is.

I would put this link into the main navigation bar, on the very right next to “Revision History”. Then I would also remove the “Home” item from the main navigation bar because it is redundant. The CKAN logo is already a link to the homepage. The only other subpage in the “Home” section is “Statistics” and that's already linked from the homepage sidebar.

But anywhere else would be fine as well.

Should return something sensible (currently reports an error).

• Create test data on visible ckan
• Discuss with JF

Update report queries with more recent versions (below are report queries shipped with trac 0.11.7):

NB: Report 1 has already been done.

id | query

1 |

: SELECT p.value AS color, : id AS ticket, summary, component, version, milestone, t.type AS type, : owner, status, : time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status <> 'closed' : ORDER BY CAST(p.value AS integer), milestone, t.type, time :

2 |

: SELECT p.value AS color, : version AS group, : id AS ticket, summary, component, version, t.type AS type, : owner, status, : time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status <> 'closed' : ORDER BY (version IS NULL),version, CAST(p.value AS integer), t.type, time :

3 |

: SELECT p.value AS color,

: 'Milestone '

milestone AS group,

: id AS ticket, summary, component, version, t.type AS type, : owner, status, : time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status <> 'closed' : ORDER BY (milestone IS NULL),milestone, CAST(p.value AS integer), t.type, time :

4 |

: : SELECT p.value AS color, : owner AS group, : id AS ticket, summary, component, milestone, t.type AS type, time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status = 'accepted' : ORDER BY owner, CAST(p.value AS integer), t.type, time :

5 |

: SELECT p.value AS color, : owner AS group, : id AS ticket, summary, component, milestone, t.type AS type, time AS created, : description AS _description_, : changetime AS _changetime, reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status = 'accepted' : ORDER BY owner, CAST(p.value AS integer), t.type, time :

6 |

: SELECT p.value AS color, : t.milestone AS group, : (CASE status : WHEN 'closed' THEN 'color: #777; background: #ddd; border-color: #ccc;' : ELSE : (CASE owner WHEN $USER THEN 'font-weight: bold' END) : END) AS style, : id AS ticket, summary, component, status, : resolution,version, t.type AS type, priority, owner, : changetime AS modified, : time AS _time,reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : ORDER BY (milestone IS NULL), milestone DESC, (status = 'closed'), : (CASE status WHEN 'closed' THEN changetime ELSE (-1) * CAST(p.value AS integer) END) DESC :

7 |

: SELECT p.value AS color, : (CASE status WHEN 'accepted' THEN 'Accepted' ELSE 'Owned' END) AS group, : id AS ticket, summary, component, version, milestone, : t.type AS type, priority, time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE t.status <> 'closed' AND owner = $USER : ORDER BY (status = 'accepted') DESC, CAST(p.value AS integer), milestone, t.type, time :

8 |

: SELECT p.value AS color, : (CASE owner : WHEN $USER THEN 'My Tickets' : ELSE 'Active Tickets' : END) AS group, : id AS ticket, summary, component, version, milestone, t.type AS type, : owner, status, : time AS created, : changetime AS _changetime, description AS _description, : reporter AS _reporter : FROM ticket t : LEFT JOIN enum p ON p.name = t.priority AND p.type = 'priority' : WHERE status <> 'closed' : ORDER BY (COALESCE(owner, ) = $USER) DESC, CAST(p.value AS integer), milestone, t.type, time

At the moment, some pages within CKAN tend to load slowly. We should create a profiling setup in which we can measure response times for complete requests and individual methods calls.

This could be used to identify bottlenecks and find an appropriate caching or tuning strategy to improve CKAN performance.

NB: We should also agree on a maximum request latency.

TODO: Read up on all those QoS tickets to avoid overlapping efforts.

Reverse engineer boto and work out how to get headers to support upload to google storage without holding api keys.

This would lead to an extension to OFS.

This analysis should inform (and go hand-in-hand) with the implementation of ticket:879 (Storage Auth API in CKAN).

• master ticket #852

Review what chunks of code we have and how to move forward with requirements.

Requirements:

• Bulk parsing is done away from the CKAN server

Possible future requirements:

• /importer for importing spreadsheets in WUI
• validation-light - 'scoring' a package

Current importers:

• /importer controller uses: PackageImporter?, form, PackageSaver?
• loader uses: package_importer -> API
• API uses: form, PackageSaver?

Modules

• package_importer:
  • xls/csv --SpreadsheetData?--> table
  • table --DataRecords?--> records
  • records --PackageImporter?--> pkg_dicts

• pkg_dict --form--> fs & validation
• fs --PackageSaver?--> package

Every change to every object is being stored in memory, which could add up to quite a lot of memory.

This fixes it by making sure the objects are in a weakref. ​https://bitbucket.org/kindly/vdm/changeset/8d5f91db641f

The tests currently take 41 mins to run on most laptops. This slows down development and discourages a test-driven approach.

We'd like to see the tests pass in 5 mins or less (but anything would be an improvement!)

Some suggestions for achieving this include:

• Upgrading the entire codebase to SQLAlchemy 0.6 so that tests could run against an in-memory SQLite database
• Not setting up and tearing down the database so frequently.