{22} Trac tickets (2647 matches)

Results (2501 - 2600 of 2647)

Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#2371 enhancement toby toby ckan-v1.9 closed fixed fanstatic integration

aims

  • allow templates to specify needed css/js
  • allow split js files to be bundled (including plugin supplied if possible - stage 2)
  • serve minified in production/ non-minified in dev
  • possibly look at doing the minification via paster - maybe as different ticket
1336046411000000 1345209675000000
#2420 enhancement toby toby closed fixed @depreciated decorator

create a depreciated decorator to wrap functions we intend to remove

logs a warning maybe check that 'depreciated' is included in the docstring too

1337858144000000 1337950422000000
#2421 enhancement aron.carroll toby closed fixed demo-theme pills break in dataset search

if pills cross onto 2 lines they split which they shouldn't to see just add some tags/formats till it breaks

1337865196000000 1343220041000000
#2433 defect toby toby ckan-v1.8 new API uses name not id for some version 3 calls

see https://github.com/okfn/ckan/pull/5

1337957648000000 1340099820000000
#2482 defect toby toby closed fixed related items dataset page 500 error 1338468524000000 1338472932000000
#2484 enhancement toby toby ckan-v1.8 closed fixed move follower functionality into helper functions 1338547770000000 1340285964000000
#2493 enhancement aron.carroll toby demo phase 5 accepted Demo site 404 needs some love

templates/error_document_template.html

I've converted the 404 page to the new style but we probably want to serve this as a 'full page' ie content is 100% page width not sidebar and primary content

just go to a non-existant url to see

Discusion:

https://okfn.basecamphq.com/projects/9558659-demo-ckan-front-end/posts/63374042/

Implementation:

http://s031.okserver.org:2375/404

1338986081000000 1343135815000000
#2496 enhancement toby toby closed fixed get debug fixed in demo branch

stacktrace

1339067709000000 1339418832000000
#2497 enhancement toby toby closed fixed abort loses language settings

if not logged in then go to add dataset url the language is lost

check if this is a ckan 1.7 issue

1339067855000000 1339421305000000
#2498 enhancement toby toby closed fixed on loggin redirect to requested page 1339067929000000 1339660986000000
#2511 enhancement aron.carroll toby closed fixed header logo should be link to / 1339513880000000 1339517817000000
#2522 enhancement aron.carroll toby closed fixed view user gatavar not sqaure

the user pic should be square on the user read page

1339604954000000 1341499095000000
#2523 enhancement toby toby demo phase 1 closed fixed New package metadata form needs creating

http://localhost:5000/dataset/new_metadata/dataset

Controller : package Action : new_metadata

template package/snippets/package_metadata_form.html

1339605176000000 1340368036000000
#2532 enhancement toby toby demo phase 1 closed fixed show activity steam for user no js

We should allow the user activity stream to be accessible with no js by passing a param or something. Need to agree how we hide/show basically agree what classes we use.

This should be a generalised solution that we can use in other places as needed.

I think this just needs discussion

1339754158000000 1340874456000000
#2536 enhancement toby toby ckan-v1.9 closed fixed cleanup activity streams 1339765323000000 1347901915000000
#2553 enhancement toby toby demo phase 2 closed fixed Create a get snippet api action

just passes template name

?template_name=...

1340020613000000 1342090078000000
#2571 enhancement kindly toby demo phase 1 closed fixed implement logic.action.create.resource_create()

This function is incomplete

If you want I can do this but I need guidance over how to do this properly. ie would I go through lib.dictization.model_save.package_resource_list_save() which looks wrong to me as I'd need the package. resource_dict_save() seems better but to be honest the total lack of docstrings etc makes me feel ill.

1340102259000000 1340369783000000
#2572 enhancement toby toby ckan-v1.9 new clean up stats plugin

attempt to disengage the stats plugin from core as much as possible

1340105054000000 1340899524000000
#2574 enhancement kindly toby demo phase 1 closed fixed look at package states

we want to be able to add new datasets but not have them indexed or seem by people

is the vdm.state the place to do this

1340115771000000 1342092416000000
#2576 enhancement aron.carroll toby demo phase 1 closed fixed resource side box in dataset resource view broken

assuming you have datahub data

http://localhost:5000/en/dataset/2008_facility_ghg_emissions_by_province_territory/resource/28650645-8878-4826-9405-f20b9dfa4c12

also the breadcrumb becomes overly long too

1340199857000000 1340617688000000
#2578 defect toby toby demo phase 4 closed fixed demo theme breaks the tests

The demo branch fails huge numbers of tests.

many are going to be template based - these need to be sorted - ideally we need to get them to work with legacy/new templates but see how we go on that one

anyhow this is not a priority whilst we are branched but needs resolving before we merge into master

1340276924000000 1347901892000000
#2579 enhancement toby toby ckan-v1.9 new move sort_by functions into lib.helpers

make these more available but keep existing functionality so not to break any users

remove_field()

drill_down_url()

etc

1340281798000000 1340899337000000
#2580 enhancement kindly toby demo phase 1 closed fixed create resource validation

david I've got the add resource working but it is happy to add empty resources which implies that it isn't doing validation correctly.

I'm not sure what the validation rules are so hard to test this - I'm coming through as an api call

let me know if i can help

1340285208000000 1342092353000000
#2581 enhancement toby toby demo phase 2 closed fixed allow state = 'draft'

pass some stuff through the context

1340287298000000 1342074635000000
#2583 enhancement toby toby demo phase 5 new make sure that we implement authentication where needed

in development many auth checks may have been lost we need to check they are still working etc

1340359478000000 1342086274000000
#2584 enhancement aron.carroll toby demo phase 1 closed fixed edit profile needs new theme

http://localhost:5000/user/edit/<my user name>

1340359925000000 1340620576000000
#2586 enhancement toby toby demo phase 1 closed fixed edit dataset - tags broken

tags is list not string

1340361630000000 1340380561000000
#2587 enhancement shevski toby demo phase 1 closed fixed author/maintainer not shown if email missing

is this the expected behaviour seems crap to me, should we just see the author but just no link

1340369450000000 1340376975000000
#2588 enhancement toby toby demo phase 1 closed fixed search index for new package

when package is added it does not seem to get indexed

1340373145000000 1340817441000000
#2589 enhancement toby toby demo phase 2 closed fixed users can view draft datasets

need to add some logic for this

1340373194000000 1342172694000000
#2591 enhancement toby toby ckan-v1.8 closed fixed postgres unicode problems postgres v9.1.4

sqlalchemy fails on postgres 9.1.4 when unicode is returned

see http://docs.sqlalchemy.org/en/rel_0_7/dialects/postgresql.html

1340627776000000 1340813161000000
#2598 enhancement aron.carroll toby demo phase 1 closed fixed stages css broken in FF

I see X where none should be

1340708102000000 1341218567000000
#2606 enhancement toby toby demo phase 1 closed fixed make fanstatic use full path

ckan/main not main

1340795985000000 1340799139000000
#2614 enhancement aron.carroll toby demo phase 3 closed fixed improve the documentation around less/node

needs to move to a better home

better ubuntu install instructions plus test they work

apt-get install nodejs

integrate with paster

1340815928000000 1343685572000000
#2618 enhancement toby toby demo phase 4 closed fixed allow nested js resources 1340878063000000 1347901864000000
#2626 enhancement aron.carroll toby closed invalid ff render issue in edit dataset

http://localhost:5000/dataset/edit/2010-nigeria-budget

the edit url button appears too high up

1341499419000000 1341499834000000
#2632 enhancement toby toby ckan-v1.9 closed fixed check allowed helpers exist

in lib helpers make sure the allowed function lists functions exist or throw an error.

1341827811000000 1342087027000000
#2633 enhancement toby toby demo phase 2 closed fixed for dev always look for templates

so we find newly created templates

1341828058000000 1341830278000000
#2634 enhancement toby toby ckan-v1.8 closed fixed change log - helper function limited in 1.8

make sure documented

1341828248000000 1342085656000000
#2638 enhancement toby toby ckan-v1.8 closed fixed add a get from request param helper function 1341844597000000 1342112772000000
#2644 enhancement shevski toby demo phase 5 assigned user dashboard for demo theme

we now have a user dashboard that needs theming not sure if we need sam to look at it

http://localhost:5000/user/dashboard

1341910821000000 1344255836000000
#2646 enhancement toby toby demo phase 2 closed fixed delete dataset

make it so

1341920466000000 1342014170000000
#2647 enhancement aron.carroll toby demo phase 5 closed fixed api bullshit button

make it as a snippet calling thing like aron would love

1341920530000000 1343229185000000
#2652 enhancement toby toby demo phase 3 closed fixed get qa working with the demo 1341937315000000 1345106904000000
#2661 enhancement aron.carroll toby demo phase 2 closed fixed delete dataset/group javascript confirm

in 2375-demo-theme-development

templates/package/confirm_delete.html needs to be made to look nice note some writing in black

delete button needs nice home on dataset/edit

you can add js support add ?confirm=yes to url on dataset button to force delete

Please note delete is not yet implemented - but will be in the next few hours

1342012395000000 1342716070000000
#2663 enhancement toby toby ckan-v1.9 new h.resource_display_name needs love

This function is shit and needs cleaning up and a doc string

description is markdown and should be treated properly

either we should truncate all or leave it to the templates but work universally

url if no name / desc this is in demo-theme branch

1342017746000000 1342017746000000
#2664 enhancement toby toby demo phase 2 closed fixed tags with : cause crazy error message on add dataset in demo 1342018503000000 1342632478000000
#2665 enhancement aron.carroll toby demo phase 2 closed fixed add dataset stages need to be form buttonable

the stages snippet needs to be inside <form> so I can make some links buttons - eg stage 1 -> stage 2 I want to save the dataset not just move to add resource

if you can just move it within the form html leaving everything else the same - I can then add buttons and pass back for styling

1342023971000000 1342088650000000
#2666 enhancement aron.carroll toby closed fixed templates/package/resource_edit.html

There is a new template for editing resources outside of the add dataset logic

this is the template it just throws some crap onto the page at the moment - we need to create a form - the form will end up with the usual data/edit stuff but for now you can use the resource to populate the form and I can then change it to use data - I just want to know which fields we'll have

templates/package/resource_edit.html

when done reassign to me and I'll get it functioning

1342084373000000 1343220176000000
#2667 enhancement shevski toby demo phase 2 closed fixed edit resource needs form

I thought I'd made this ticket before but can't find it. When we have a fully created dataset we need to be able to edit the resources.

template: package/resource_edit.html

when you've made a form reassign to me and I'll get it working

1342170103000000 1343058578000000
#2669 enhancement aron.carroll toby demo phase 5 closed fixed style [Draft]

I've added [Draft] to partial completed datasets in the package list and read commit 6a8e74d0

templates/snippets/package_list.html

templates/package/read.html

do you want to style them in some way?

1342174110000000 1343907572000000
#2671 enhancement toby toby ckan-v1.9 closed fixed fix allow_partial_update

need to be able to delete stuff explicitly

1342183618000000 1345209579000000
#2684 enhancement aron.carroll toby demo phase 2 closed fixed .js leaving forms warning

I think it would be nice to have all form pages monitor their forms - excluding search ones - and if a field has changed then warn the user that they have not saved the form data but are leaving the page.

This would help with things like changing resources on the add dataset resource form etc.

1342510477000000 1342539637000000
#2685 enhancement toby toby demo phase 2 closed fixed delete buttons should be form buttons

stop bots/browsers etc following links and auto deleting stuff

1342510565000000 1342514443000000
#2693 enhancement seanh toby ckan-v1.8 closed fixed License is out of date

https://github.com/okfn/ckan/blob/master/LICENSE.txt mentions requires/lucid_conflict.txt which is no longer the case for 1.8 and needs updating

not sure where requirements-testing/docs fit in

@sean I've assigned this to you as you changed the requirements and my plate is full feel free to pass on to someone else

1342605447000000 1343148356000000
#2694 defect toby toby demo phase 2 closed fixed index page 500 error

we need to have proper fix for this - not sure if there is some caching issue

this around the group search

1342605699000000 1342680771000000
#2708 enhancement kindly toby ckan-v1.9 new limit extra data for package/group show

contextpackage_limits? = { 'tags': 5, <- get first 5

'extras': 0, <- get all

}

only get what you ask for have to be explicit

contextgroup_limits? = {} only main item

start with datasets/groups expand if we like it

1342622420000000 1342622420000000
#2710 enhancement tobes toby demo phase 3 closed fixed make the api_info.html not look like shit

you know the score

1342624385000000 1343318314000000
#2711 enhancement johnmartin toby demo phase 5 closed fixed bin/less dies for toby

it would be nice if this didn't fall over when the dir is removed maybe a linux only issue?

1342631508000000 1352205968000000
#2712 enhancement toby toby demo phase 2 closed fixed delete resource functionality needed

we need a delete_resource logic action and enable for resources

1342680517000000 1342690104000000
#2713 enhancement aron.carroll toby demo phase 3 closed fixed button colour not set properly in variables.css

to reproduce run

paster color fuchsia -c development.ini

then go to http://localhost:5000/dataset/new

Add button is blue gradient at the top this should be overridden in variables.less

when added send this ticket to me and i'll update the color command to change it

1342706428000000 1343656865000000
#2717 enhancement shevski toby demo phase 3 closed fixed French breaks the demo header

when language is french

a) the menu overlaps the logo

b) much of the logo becomes unclickable

@shevski

a) seems like you b) is aron

1342780487000000 1344359846000000
#2741 enhancement johnmartin toby demo phase 4 closed fixed admin index styling

http://localhost:5000/ckan-admin

you need to be an admin

paster sysadmin add <username> -c dev.ini

we need to get links to other admin pages etc - I think this will want to be flexible so extensions can add extra links etc - but that can come later

1343124308000000 1352205988000000
#2742 enhancement johnmartin toby demo phase 5 closed fixed admin authz styling

http://localhost:5000/ckan-admin/authz

you need to be an admin

paster sysadmin add <username> -c dev.ini

like #2741 we need other admin links

you may want to make the inputs use your form macros - I've just done a basic port

also we will need to get all the tables done but looks like we need to get auth groups working for that

1343124423000000 1352206081000000
#2744 defect aron.carroll toby demo phase 2 closed fixed license dropdown is cramped

http://s031.okserver.org:2375/dataset/new

license dropdown - when open we need some line spacing at least in FF I think it'd be nice to fix this in phase 2

1343144543000000 1343219298000000
#2749 enhancement aron.carroll toby demo phase 5 closed fixed page title issue with super() super()

title broken for

http://s031.okserver.org:2375/en/dataset/new_resource/gold-prices

we have inherited the title from the extended page not from the base

should be 'add data - ckan' not 'add data - create dataset - ckan'

any idea how we fix this?

1343215828000000 1343232304000000
#2750 enhancement toby toby demo phase 3 closed fixed javascript translations need ripping

we need to ensure that translations in .js files are ripper for translation

look at what is available as preference to creating our own

1343216246000000 1343415279000000
#2751 enhancement toby toby demo phase 5 new check translations for full demo site

need to check everything gets translated - sean did this before so will have info

1343216443000000 1344243046000000
#2758 enhancement toby toby ckan-v1.9 new file storage gives error if config not available but no useful user information

We get an error which should be improved the actual problem is this but not passed to user

KeyError?: 'ofs.impl'

View as: Interactive (full) | Text (full) | XML (full) Module ckan.controllers.storage:2 in auth_form view Module ckan.lib.jsonp:26 in jsonpify view

data = func(*args, kwargs)

Module ckan.controllers.storage:407 in auth_form view

authorize(method, bucket, label, c.userobj, self.ofs)

Module ckan.controllers.storage:200 in ofs view

StorageAPIController._ofs_impl = get_ofs()

Module ckan.controllers.storage:71 in get_ofs view

storage_backend = configofs.impl?

Module paste.registry:146 in getitem view

return self._current_obj()[key]

KeyError?: 'ofs.impl'

1343287709000000 1343287709000000
#2759 enhancement aron.carroll toby demo phase 3 closed fixed resource upload pointer incorrect

On the add resource forms new or existing dataset

when hovering over Upload a file css cursor should be pointer not text

1343289469000000 1343651082000000
#2778 enhancement aron.carroll toby demo phase 3 closed fixed apps dashboard needs some design love

apps dashboard available at http://localhost:5000/apps in main dev branch

I have got this to work in the demo however

a) it is a bit ugly

b) the template could do with some sorting eg the dropdowns should be being created in a nicer way

1343734040000000 1343841130000000
#2789 enhancement aron.carroll toby demo phase 3 closed fixed logo fixups

With the text logo I'd like the tag line to be under the site name so some css like

float:left;clear:left;font-size:13px;line-height:0.1;

Also the site name can be larger if no tagline

commit daac54dbd856 I've added a new global g.header_css that sets the hgroup class to

'header-image' 'header-text-logo' 'header-text-logo-tagline'

I think maybe we should also have 'header-image-css' for the equivalent of the class="logo" in the link to allow/stop the image replacement

1343835141000000 1343835904000000
#2790 enhancement kindly toby demo phase 4 new logic.action.user_show is slow

This is a very slow call it would benefit from the sort of speed-ups that package-search received

for me locally this is taking 6 seconds for rufus using the datahub data I have. I think a lot of this is the dataset retrival/dictization

can we just grab json blobs from solr?

also is it possible to specify a sort order/paging?

I've put this as a demo-theme ticket as it is an big issue on the demo we are at 25 second page loads - which i can get down to about 8.8 secs so this is the main pain point now

1343852483000000 1345023734000000
#2791 enhancement aron.carroll toby demo phase 3 closed fixed Logo fix

@aron

We should be able to have longish tag lines that look ok currently they wrap and stop the logo text being clickable

Can we have white-space:nowrap; for the tagline so long ones don't wrap and maybe look at aligning long ones

1343895034000000 1343906891000000
#2792 enhancement aron.carroll toby demo phase 3 closed fixed dropdown fixes - open correctly

http://s031.okserver.org:2375/ language dropdown expands downwards it should open upwards

select2 can do this see http://ivaynberg.github.com/select2/#documentation can you see if we can get this in ckan demo

1343895296000000 1343901501000000
#2794 enhancement aron.carroll toby demo phase 3 closed fixed top buttons missing on edit resource

http://s031.okserver.org:2375/dataset/resource-markdown/resource_edit/ba1b094d-697c-47c4-a72a-6c2557da5409

should have back to resource as a minimum

1343897926000000 1343903064000000
#2795 enhancement toby demo phase 5 new Check validation of HTML, CSS, JS

Ensure that we are being standards compliant

1343903128000000 1343903128000000
#2797 enhancement aron.carroll toby demo phase 3 closed fixed plain_textarea styling

http://localhost:5000/ckan-admin/config uses a plain (non markup) textarea for custom css

I've added form macro plain_textarea to provide one, however it renders without a bottom border due to markup textarea styling can you fix this - thanks

1344000422000000 1344250973000000
#2798 enhancement toby toby demo phase 4 closed fixed Admin Config reset

this want to

a) be a link on the left of the form like deletes

b) needs a confirm like delete

c) wants to work with the autoform

1344012605000000 1345540010000000
#2805 enhancement seanh toby closed fixed zh_TW has no Plural-Forms

the .po file for zh_TW does not contain Plural-Forms which breaks the i18n JavaScript? Extractions for ckan-demo.

I've added the following which may or may not be correct - from web search "Plural-Forms: nplurals=1; plural=0\n"

I suspect it will get overwritten when translations are pulled from transiflex so passing to you as you are the expert in that realm

1344253863000000 1344254760000000
#2807 enhancement aron.carroll toby demo phase 3 closed fixed autocomplete.js error

@aron,

edit dataset has a js error

http://localhost:5000/dataset/edit/1-kmwaterfractionfromnationaltopographicdatabasemapscanada

TypeError?: options is undefined [Break On This Error]

this.lookup(options.term, options.callback);

autocomplete.js (line 231)

1344266228000000 1344270190000000
#2808 enhancement aron.carroll toby demo phase 3 closed fixed Confirm delete dialog badly placed on small screen

If you make your browser window short - say half screen height on a page with delete eg http://localhost:5000/dataset/edit/1-kmwaterfractionfromnationaltopographicdatabasemapscanada

the dialog box is partially off the screen - it should try to stay on screen

1344333474000000 1345023693000000
#2809 enhancement aron.carroll toby demo phase 3 closed fixed _ not defined for modules

eg on confirm delete

http://localhost:5000/zh_TW/dataset/edit/1-kmwaterfractionfromnationaltopographicdatabasemapscanada

_(....) gives [object Object] not translation

1344335863000000 1344347814000000
#2816 enhancement toby toby ckan-v1.8 closed fixed DomainObjectOperation not in model

Over-enthusiastic removal of some model items

this breaks qa extension

1344496621000000 1345040471000000
#2822 enhancement toby toby demo phase 4 new Resource additional info titles format/i18n

the title for additional info should be translated

capitalised etc

1344504620000000 1344543985000000
#2823 enhancement toby toby demo phase 5 new resource additional info title order

Order the items so that none user fields are first from ticket #2707

1344504773000000 1344504773000000
#2829 enhancement johnglover toby ckan-v1.9 new Archiver fails on 403 http response

Had this issue with the archiver on my local machine need to be logged in (I am admin) to see via web front end

$ paster archiver update -c ../ckan/development.ini 


2012-08-09 11:01:37,636 INFO  [ckanext.archiver.commands] Archival of dataset resource data added to celery queue: opencontext-chogha-mish-fauna (1 resources)
2012-08-09 11:01:37,671 INFO  [ckanext.archiver.commands] Getting dataset metadata: south-african-national-gov-budget-2012-13
2012-08-09 11:01:37,900 INFO  [ckan.lib.base]  /api/action/package_show render time 0.043 seconds
Traceback (most recent call last):
  File "/home/toby/okfn/pyenv/bin/paster", line 8, in <module>
    load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py", line 104, in run
    invoke(command, command_name, options, args[1:])
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py", line 143, in invoke
    exit_code = runner.run(args)
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/script/command.py", line 238, in run
    result = self.command()
  File "/home/toby/okfn/pyenv/src/ckanext-archiver/ckanext/archiver/commands.py", line 98, in command
    response = app.post(api_url + '/package_show', data)
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py", line 262, in post
    expect_errors=expect_errors)
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py", line 243, in _gen_request
    return self.do_request(req, status=status)
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py", line 406, in do_request
    self._check_status(status, res)
  File "/home/toby/okfn/pyenv/lib/python2.7/site-packages/paste/fixture.py", line 439, in _check_status
    res.body))
paste.fixture.AppError: Bad response: 403 Forbidden (not 200 OK or 3xx redirect for /api/action/package_show)
{"help": "Return the metadata of a dataset (package) and its resources.\n\n    :param id: the id or name of the dataset\n    :type id: string\n\n    :rtype: dictionary\n\n    ", "success": false, "error": {"message": "Access denied", "__type": "Authorization Error"}}
1344508484000000 1344508484000000
#2830 enhancement toby toby demo phase 4 new Need method to undelete groups

need controller action and front-end method

1344509408000000 1344547341000000
#2836 enhancement toby toby demo phase 3 closed fixed Need extra info for forms

like we have on the form here

http://s031.okserver.org:2375/harvest/new

you said it partly exists so maybe I just need some training

1344605931000000 1345023663000000
#2837 enhancement aron.carroll toby demo phase 3 closed fixed checkbox styling

Is this done I know checkboxes are a pain but it looks a bit out of place compared to the other items.

1344605993000000 1344851271000000
#2840 enhancement aron.carroll toby demo phase 3 closed fixed tag line styling

@aron

Is it possible to get the tagline text to float the other way

at the moment we have

...........Title Tag line to here

I'd like

...........Title ...........Tag line to here

is this possible or too much pain

based partly on this comment Markw wrote

A minor problem visible on the home page <http://s031.okserver.org:2375/> and other pages, when viewing in both Chrome and Firefox:

if the browser width is not very wide, the tag line 'Open source data portal' vanishes off the *left* hand side of the screen. For some reason the browser doesn't even recognise this with a horizontal scroll bar (as it does when stuff is off the *right* hand side) - it just chops off the text.

1344851307000000 1344852538000000
#2858 enhancement toby toby demo phase 3 closed fixed Harvester tweaks

I've had a look at the new harvesting theme, it looks great, thanks. I did some minor changes and reworded the errors as per tickets #2852 and #2853, but I kept the messages red, as the only other ones available ("flash_notice") are blue and are not very prominent.

Toby, on the old form there was a list of descriptions for all available harvesters which could be nice to have back. I couldn't find an easy way with the new form macro, so I don't know how easy would be to implement

http://i.imgur.com/pzS8z.png

1345103198000000 1345105842000000
#2875 enhancement toby demo phase 4 closed fixed Fanstatic css issues

Fanstatic does not support

@import ...

relative imports for images etc

need to get a fix for these asap

1345208199000000 1345214469000000
#2876 enhancement toby toby demo phase 4 closed fixed Admin Config changes are not forced

need to make sure these update everywhere when done

1345208451000000 1345470265000000
#2939 enhancement toby toby new Orgs are groups

holder for ticket number

1348564034000000 1348564034000000
#3012 enhancement toby toby new data.gov auth 1352393409000000 1352393409000000
#3030 enhancement toby new clean up helper functions 1355834038000000 1355834038000000
#1120 enhancement tsm ckan-backlog new Atom feeds of each tag

Tags could/should have an Atom feed. This would mean that every edit to relevant packages could be easily monitored. See [1].

[1] http://lists.okfn.org/pipermail/ckan-discuss/2011-May/001162.html

1304309285000000 1339774568000000
#318 defect johnglover wwaites ckan-sprint-2011-10-28 closed fixed Insufficient validation of resource URIs

The CKAN instance on data.gov.uk serves invalid URIs out of its API.

For example the following can be found,

http://uk.sitestat.com/homeoffice/rds/s?rds.hosb0509tabsxls&ns_type=pdf&ns_url=[http://www.homeoffice.gov.uk/rds/pdfs09/hosb0509tabs.xls]

In this URI, the : and / characters after the ? in the query part are invalid according to section 3.4 of RFC2396

Also URIs are not stripped of whitespace at the end.

This causes problems when other software with a more correct interpretation of what a valid URI is attempts to consume data from CKAN. In this instance the Talis triplestore complains about such URIs.

"Be liberal in what you accept and conservative in what you send" would seem apt.

Actions

  • Validation of urls as part of form entry or data loading
    • Need to consider situation where this should happen out-of-band (i.e. we allow load even with invalid data and then flag bad dates in separate validation process). In general doubtful that we should do this here because url invalidity is such a big deal
  • This code should support analysis of existing data so we can go through existing database and find invalid urls
    • Also useful to have this so we can do out of band validation
1274302723000000 1349778662000000
#433 defect wwaites datapkg-0.8 closed fixed Data package metadata in the Egg

Still not sure if we shouldn't use the existing setuptools machinery to manage this -- there is already a way to get at the metadata. In any event, I've just made an addition to datapkg that makes it possible to put datapkg_sources as a dictionary in your package's setup.py. Afterwards it is possible to pull the metadata out of the egg. Of course this could easily be changed to save the information in whatever form, indeed if you pass it a string instead of a dictionary it will just write whatever you gave it into the datapkg_sources.spec. The point is, I think that the egg is a good place to stuff this information.

For non-python users, it is always possible to simply put up the datapkg_sources.spec somewhere on the web so they can directly retrieve the data files.

From the docstring::

    This is the implementation for an [egg_info.writers] entrypoint.
    Datapkg adds an argument to setuptools's setup() function called
    datapkg_sources. The argument should be a dictionary of the form:

    .. code-block:: python

        setup(
            ...,
            datapkg_sources = {
                "cra2009" : "http://www.hm-treasury.gov.uk/d/cra_2009_db.csv"
            }
        )

    The result of this is that there will be a file in the egg called
    datapkg_sources.spec that looks like this::

        [sources]
        cra2009=http://www.hm-treasury.gov.uk/d/cra_2009_db.csv

    How do you get at this data? Simple::

    .. code-block:: python

        import pkg_resources
        dist = pkg_resources.get_distribution("ukgov_treasury_cra")
        spec = dist.get_metadata("datapkg_sources.spec")

    and 'spec' will be the contents of the file as a string.
1282078660000000 1283183647000000
Note: See TracReports for help on using and creating reports.