{23} Trac comments (3729 matches)
Results (2401 - 2500 of 3729)
| Ticket | Posixtime | Author | Newvalue | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| #1400 | 1331544485000000 | rgrp | Hmmm. Emerging consensus from discussions in last few months is that wiki is fairly confusing and we should move core, solid docs (from whichever area) to the main docs. Furthermore, key dev info like how to setup and do extensions *is* already in the main docs (see http://docs.ckan.org/en/latest/writing-extensions.html) and has been for 6m+. I've created #2226 for more work on docs and you may wish to comment there. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2454 | 1339493727000000 | aron.carroll | Hmm, I totally agree with you on the missing username front but it would be nice at least to provide the errors in the same way we do the other forms. Currently it just displays a flash message which is unexpected. I'd suggest we add a generic "failed to login, invalid fields" in the error_summary block. Then more specific "this field is required" when a required field is left empty. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1401 | 1320142359000000 | rgrp | Hi, this is very useful to hear. Can you point out links or even better submit a patch / pull request: http://wiki.ckan.org/Submitting_a_code_patch | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #936 | 1298283172000000 | thejimmyg | Hi Wayne, I'm assigning this to you but it isn't a priority yet. We'll put it in a sprint when it is time to do it. Cheers, James | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1197 | 1340018903000000 | seanh | Hi Toby, yes I think I agree with you, it reminds me of when I install new Android apps on my phone, some of them start off by bothering me with some sort of guided tour or long instructional text, I hate this and always skip through without reading it, expecting to be able to figure out the interface for myself. The one thing that something like this could be useful is, rather than showing someone how to use the site, showing them what the site is capable of so they can assess whether it meets their needs/is something they're interested in. However, we already have a good guided tour of this type on ckan.org/features, those pages are really pretty good. Close this as wontfix? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #447 | 1294410666000000 | thejimmyg | Hi Richard, could you take a look at this please? It is probably invalid now, in which case feel free to close it. Many thanks, James | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1150 | 1311178163000000 | thejimmyg | Hi John, Can you please check that the new webstore+preview works correctly with this one too please? Cheers, James | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1062 | 1311178145000000 | thejimmyg | Hi John, Can you please check that the new webstore+preview works correctly with this one please? Cheers, James | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1183 | 1311181719000000 | thejimmyg | Hi John, Can you have a look at this one too please? Cheers, James | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1175 | 1314030683000000 | dread | Hi Flavio, is this still a concern? I'd like to close this ticket unless you can provide details of the error from your logs or exception email. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2722 | 1343060105000000 | shevski | Hey, I only mentioned bug 2 here because I'm pretty certain this already exists as a separate ticket. Have definitely reported it before, but i think someone else created the ticket - so just wanted to remind that this hasn't been fixed either yet. What's the best way to do this in future? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1683 | 1326976925000000 | dread | Here's someone else with the exact sorting problem: http://lucene.472066.n3.nabble.com/last-item-in-results-page-is-always-the-same-td2513423.html | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #318 | 1349778662000000 | dread | Here's a real example - one of many from MOD {{{http://www.dasa.mod.uk/applications/newWeb/www/index.php?page=48&thiscontent=180&date=2011-05-26&pubType=1&PublishTime=09:30:00&from=home&tabOption=1}}} Browsers accept colons and slashes happily, which is the main usage of our links. The URL looks better with the colons and slashes, rather than the encoded version. The average departmental user doesn't understand that the reason to encode them is for some academic RFC and RDF which is not "liberal in what it accepts". Since the RDF tool has a satisfactory way to encode links, this problem is essentially solved. Therefore I'm changing ckanext-archiver to accept these unencoded links, I'm afraid. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1004 | 1323187352000000 | dread | Here is the unclipped version: {{{ <p> <span i18n:msg="" class="ckan-logged-in" style="display: none;"> <a href="${h.url_for(controller='group',action='new', id=None)}">Create a new group</a> </span> <span i18n:msg="" class="ckan-logged-out"> To create a new group, please first <a href="${h.url_for(controller='user',action='login', id=None)}">log-in</a>. </span> </p> }}} {{{ <li py:attrs="{'class':'current-tab'} if c.action=='new' else {}" class="ckan-logged-in" style="display: none;"> ${h.subnav_link(c, h.icon('group_add') + _('Add a Group'), controller='group', action='new')} </li> <li py:attrs="{'class':'current-tab'} if c.action=='new' else {}" class="ckan-logged-out"> ${h.subnav_link(c, h.icon('group_add') + _('Log-in to add a Group'), controller='group', action='new')} </li> }}} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #698 | 1292596589000000 | Stiivi | Here is the fork for (json) data proxy: https://bitbucket.org/Stiivi/dataproxy I've refactored it and moved transformations into separate modules. For each resource type there should be a module in transform/<type>_transform.py Each module should implement ``transform(flow, url, query)`` and should return a dictionary as a result. Existing modules: * transform/csv_transform - CSV files * transform/xls_transform - Excel XLS files if there is no resource_type module, HTTP 200 Error Resource type not supported is returned. You can override URL file extension or specify type if extension is missing through type= URL option. For example if you have any URL that contains CSV data however the url is just foo.com/data then you can pass: url=http://foo.com/data&type=csv Note: Source refactored/updated in example/dataproxy, being tested by running locally localhost:8000. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1040 | 1328806824000000 | dread | Here are all the CKAN instances potentially involved in this issue: == ckan.net / DGU == These suffered this problem briefly and were fixed 15/3/11 == opengov_es http://opengov.es/ 1.2 == This does not have a theme extension. Is not serving files from root. == nosdonnees_fr http://www.nosdonnees.fr/ 1.3.1a == This does have a theme extension but does not seem to have a trailing blank public path since it is not serving files from root. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #881 | 1294410574000000 | thejimmyg | Hello fccoelho, Could you please post the output you get from running pip? Thanks, James | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2445 | 1338472966000000 | ross | He didn't say but I assumed yes given current impl uses dodgy JS->API. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #972 | 1310134129000000 | thejimmyg | Having thought about this more David Raznick and I have decided to stick with separate extras for the timebeing. Mainly because of internationalisation issues. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1737 | 1336328947000000 | rgrp | Having now thought about this I'm re-opening this ticket for the following reasons: * No real documentation (other than that in this ticket yet available) * It would also be nice to know how this maps to SOLR API (can i use all of the facet options solr provides or not ...?) * And I would again emphasize my preference for having *direct* access to something that looks *exactly* like SOLR API as I can then use client and docs from SOLR to work with it. * No clear resolution of separation between Action and REST API (and search API). Really seems to me there should be convergence between latter 2 (as suggested in the ticket) -- this would also resolve the problem that having GET /api/dataset return all datasets is *not* a great idea * The Action API requires a POST request. Since the primary purpose of the search API would be usage from JS it would be nice if GET and JSONP were supported. (Though given our CORS support we could argue this was optional). Not saying *all* of this needs fixing but some clear approach here would be useful | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #864 | 1291741028000000 | memespring | Havent implemented all text changes because worried about impact on other themes. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #435 | 1298284084000000 | thejimmyg | Haven't seen this myself and it is 6 months old now. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #463 | 1294916148000000 | dread | Haven't seen this for ages. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1451 | 1321876014000000 | johnglover | Haven't looked at this yet, moving to new sprint | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #447 | 1294410738000000 | memespring | Have you got a link or screengrab? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #833 | 1295263447000000 | rgrp | Have repo https://bitbucket.org/okfn/ckanext-admin | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2513 | 1340187230000000 | ross | Have pushed back to 1.9 but if may be the the converter service will replace it in the meantime. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1761 | 1328703769000000 | ross | Have overridden the history function in the DGU package_gov3.py controller to make sure the viewer is in at least one group. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #364 | 1291637291000000 | rgrp | Have now switched to solr search (and maybe working in postgres by now). Note correct link is http://ckan.net/package?q=statistics | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1003 | 1298889293000000 | rgrp | Have now started refactor to use backbone and have basic inline editing working and started on Add dataset view. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1296 | 1315821293000000 | rgrp | Have now done resource add pretty effectively and most of edit done (should complete today). | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1688 | 1327315514000000 | dread | Have not really started this - rolling into new sprint. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1460 | 1328638393000000 | dread | Have not looked at this yet. Bumping to this sprint. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1446 | 1321872724000000 | rgrp | Have made significant progress on new recline but not yet operational. See https://github.com/okfn/recline and https://github.com/okfn/recline/issues/6 https://github.com/okfn/recline/issues/12 https://github.com/okfn/recline/issues/8 also https://github.com/okfn/recline/issues/10 and more ... | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2223 | 1331558791000000 | rgrp | Have introduced boostrap js and css on merge of #1797 https://github.com/okfn/ckan/commit/476a5bd32a3fac5d2dd85614f5d86c79f4ff6547 However need to remove blueprint and make sure we keep our original theme. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #50 | 1239018913000000 | rgrp | Have done a bit of digging but noot currently a priority given direction datapkg is going. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2651 | 1342440899000000 | ross | Have assigned to icmurray to test | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2814 | 1346670381000000 | ross | Have assigned to client team to decide what they want to do about this. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2302 | 1342097600000000 | ross | Have assigned it to Toby. Not sure how high priority this is, but only provides a very minimal level of configurability. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2619 | 1346670257000000 | ross | Have a push request with a privacy flag which might be useful in determining the status of the dataset (public/private) but no idea if it will be accepted yet. Have assigned back to you for safe-keeping so it doesn't get lost in backlog. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #448 | 1286386256000000 | dread | Has this all been finished yet? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2440 | 1339421746000000 | aron.carroll | Has some styling as of b0153ba but needs content to be applied. Toby, can you pull in the top 10 tags, as well as n of the most "popular" datasets and n of the most recent datasets. Actually just providing the functionality as helpers would be great. Do we actually have "featured" datasets implemented at the moment? Not sure what we're going to use for the box on the top left when on a new base CKAN instance. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1433 | 1324472583000000 | dread | Has gone into CKAN 1.5.1 release. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #406 | 1296467427000000 | pudo | Has existed for a while, not implementing any more except for what is in #852. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2818 | 1346231718000000 | ross | Has been suggested that we are clearer on why the URL is required, as this related items are really just 'Links'. We should consider the name change, and also possibly make the types configurable in config somewhere as they are currently almost content free. Assigned to Sean to keep an eye on this ticket until we get a reasonable chance to action it. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #522 | 1287392999000000 | pudo | Has been fixed for quite some time. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #49 | 1214244368000000 | rgrp | Has become less of a problem since big efforts a few months ago so this can be downgraded for time being. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2654 | 1343126740000000 | ross | Has a dependency on the new themes. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2699 | 1343597746000000 | shevski | Groups & auth notes: Datasets have an owner Dataset owner must be a group (not users) Actions on dataset are: Create: any logged in user who is a member of a group may create a dataset Choose which group will be owner on create page Read (determined by public/private flag on dataset). Private means only readable by members of owner Group Edit: any member of the owner group Admin: (change state, change owner) only can be done by Admins of owner group Resources: inherit permissions from dataset Groups: Create: Configurable whether creatable by any logged in user or just by sysadmins Edit: only group admin Read: public/private flag on group (?) Admin: (delete, update members / admins) Admin of Group Groups have users with roles/capacities on the group Admin Member System-wide sysadmin (boolean flag on your user account) who can do anything | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2569 | 1340878031000000 | shevski | Great! Can we change it so that the resource name (if available) displays instead of the URL? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1400 | 1331481461000000 | rgrp | Great to see this is done. I do have one question: why did we put this on the wiki rather than in the proper CKAN docs? In general, I think it would be better to use the wiki less and our proper docs more ... | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1134 | 1305125228000000 | thejimmyg | Great stuff. Yes, I agree with all this, but as initial thoughts I suggest that for phase 1, the info() method returns one more key called "form_config_interface" which takes a string representing a the type of interface for the config field on the form. If the key is missing it is treated as "None". The two possible values are: None - the field is not present and must always be stored as NULL Text - a single text field will be provided on the interface Whatever the interface, the value built will always be stored in the config column of the table. We then also provide a "get_schema()" method that returns a schema cablable of parsing the data submitted by the form and storing it as a single key named "config". ckanext-inspire will then add other functions to the schema for URL etc so that regardless of what the harvest plugin does, the key fields ckanext-harvest are always there. This then allows: * Customiseable config interfaces * Customisable valiation * A consistent place to store config in the database Sound good? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1004 | 1323195485000000 | dread | Great stuff zeph! Diff looks really good and cheers for the explanation. On master, cset:a89a48731ba548170045a60ac2930e0019c299c7 and I've cherry picked this for release 1.5.1. too. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1065 | 1302081747000000 | dread | Great stuff Seb, cheers for that. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1129 | 1305116419000000 | thejimmyg | Great proposal, here's my thinking: * The current VDM model (where we have things like a single package table which logs changes to a package_revision table) is very well tested and works well and we should only consider a radical change with good reason * Although we are making a move from *model objects* being the central components to the *logic layer functions* being the central components, there is no need yet to have the database structures we store (for revisioning or otherwise) start to look like the dictized representation. The logic layer itself is designed to do that mapping. * Since we do have this logic layer though, there is less need for "magic" SQLAlchemy objects to support dicts and lists (eg the stateful objects used to make a list of tags on a package behave like a list), a few simple (and easily debuggable) queries in the logic layer would work better. In the longer term we may want to look at serialising more dictized-looking objects and that may lend itself to a more dictized changeset model or even a more dictized storage system (eg no-SQL) but for the time being we are not at that point. I recommend the following: * We continue to use the current default branch of VDM (not the changeset branch) * We continue to treat the package table (and other non-revision tables) as the most recent revision (even though the *active* revision displayed in CKAN might well be in the revision tables because more recent changes haven't been moderated yet) * We slowly stop using stateful lists and dicts in CKAN because we have move control with explicit queries in the logic layer Sound good? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1497 | 1322569805000000 | dread | Got rid of this one that appears on many CLI commands: {{{ /home/dread/gitroot/pyenv-ckan/lib/python2.6/site-packages/pylons/templating.py:610: UserWarning: Unbuilt egg for setuptools [unknown version] (/usr/lib/python2.6/dist-packages) Engine = entry_point.load() }}} cset:e1e2918 Look out for others and reopen when found. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2714 | 1342794133000000 | shevski | Got it working http://demo.ckan.org/dataset/test-dataset/resource/14244c9c-7dcd-43f0-9df0-b363baa2fa8d | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #501 | 1282724566000000 | dread | Good thinking - see previous ticket on this: ticket:441 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #47 | 1201600216000000 | rgrp | Good suggestion (this has been on my mind for a while). This should be pretty cheap to do, see instructions at: http://wiki.pylonshq.com/display/authkitcookbook/OpenID+Passurl We probably just want to create a normal genshi template and then have a function that renders which can then be set in the config file. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #659 | 1298424109000000 | nils.toedtmann | Good idea. Listed this in my nagios ticket http://knowledgeforge.net/okfn/tasks/ticket/600 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2322 | 1339750208000000 | ross | Going to try that last comment again. No, no new features in point releases, we only put bug fixes in unless there is an extremely pressing need. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2736 | 1346670097000000 | ross | Going to be reworked without celery. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1275 | 1319812967000000 | dread | Going into release 1.5. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1371 | 1324037905000000 | dread | Goes into release 1.5.1. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1319 | 1324036947000000 | dread | Goes into release 1.5.1 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #926 | 1298541597000000 | anonymous | Goals: We want the interface for updating an object to be loosely coupled to the method for updating it. We might update a Package from: - HTML forms - a REST API (using JSON) - a CLI (potentially using command line arguments, YaML, XML or ini files) Right now, data is validated using a form framework, even if we're not using forms. Data is written to the object as part of the forms framework (using the "sync()" method), making the process hard to customise and hard to discover. Instead, there should be a standard chain for: - deserialising untyped data (such as that received from an HTTP POST or parsed from a YaML file) into valid data - returning structured errors suitable for displaying to the user - saving the validated, deserialised data Ideally, it would look something like: schema = MySchemaDefinition() raw_data = open("raw.csv", "r").read() structured_data = to_python(raw_data, schema) try: validated = validate(python_data) myobject.update_from_dict(validated) return "Updated OK" except ValidationError, e: return "Error: %s" % e.to_dict() The inverse would be something like: structured_data = myobject.render_to_dict() raw_data.write(to_csv(structured_data, schema) print "Wrote CSV %s" % to_logformat(serialized_data, schema) The question of how to generate and display forms should be completely decoupled from this. It should be easy to write forms by hand, which means it should be simple to flatten the serialized data to key, value pairs, and match up any validation errors to each key. Optionally, a form widget generation framework is a nice-to-have, but not essential, as it is expected that, given enough time, the majority of forms will require manual coding to accomodate edge conditions. A form widget generation framework should be reasonably complete if it's worth trying at all, which means it should support things like: - nested fields (at least repeating, multi-value fieldsets) - widgets for dates and file uploads - internationalisation ...but note I'd settle for *no* widget generation Components of a serialisation / validation framework: - a simple, obvious way to define a schema - a lightweight validation implementation - simple interface for validators - easy to match validation errors to data structure items Overall, I'd like to see: - loose coupling, no framework dependencies - maximal test coverage - extensive documentation with readily available examples ## Findings I looked at flatland, formencode, FormAlchemy, formish, WTForms, Django, web2py, deform/colander, formconvert and web.py - **web2py** just helps build HTML from python, so isn't what I'm after at all - **web.py** has rudimentary validation which is only aimed at HTML forms and is hence tightly coupled with them. - **Django**'s forms are again tightly coupled to HTML forms (and their generation) - **FormAlchemy** similarly couples validation to forms, and is focussed on inferring a schema from a data model SQLAlchemy. - **WTForms** again focuses on Form generation and don't make itx easy to deserialise arbitrary data This leaves us with Flatland, Formencode, Formish, Colander/Peppercorn/Deform, and FormConvert. Having reviewed all of these, I rejected Formencode on the basis of its patchy documentation and relatively low unit test coverage. I also found it mixed concerns a bit much for my taste. Formish felt similarly sparsely documented. Of the remainder, I'd be happy using any of them, but opted for Colander in the end as it has the most exhaustive documentation and unit tests and has been used in production for a long time. FormConvert has a nice design but is a bit of a moving target at the moment -- worth revisiting in the future. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2204 | 1335795665000000 | ross | Giving this ticket to Aron to fix the front end | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2317 | 1340033433000000 | kindly | Getting replace as part of new theme. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #128 | 1275694573000000 | Floallgloli | Futures Trading Key your Futures education provider Learn how to trade futures with live trading education Futures education and trading free trial live on line room <a href=http://www.futurestradingkey.com><B>Futures</B></a> | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1518 | 1323761438000000 | rgrp | Further investigation shows that: * Spaces and any other characters work fine with local file storagfe * Google escapes other non-url allowed characters (so should work fine) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2442 | 1338302281000000 | aron.carroll | Functionality to be added: * Group module needs to display group/publisher title, logo and description if dataset has an associated group. * Popular icon only needs to be displayed if dataset is popular. (say if it has 10+ views for the moment) * Popular icon needs localised text passed in to snippet (See TODO comment above helper https://github.com/okfn/ckan/blob/feature-2375-demo-theme/ckan/templates/package/read.html#L46) * Related module needs to show most recent item or fallback content. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #223 | 1264439367000000 | dread | From rgrp: Looked through jquery plugins and best I could find was this one: http://addywaddy.github.com/jquery.tagcloud.js/ ( http://plugins.jquery.com/project/jquery_tagcloud) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1435 | 1323283538000000 | dread | From IRC today: {{{ <rgrp> dread: btw have a big suggstion -- switch to continuous.io for our buildbot stuff ... <rgrp> openspending have done this and it's a nice setup ... <dread> rgrp: interesting - what's better? <dread> i assume it just starts a script and reports the result? <rgrp> that's right though also has integration with some bakcens (but we probably don't need that) <rgrp> the point is we don't need to boot machines, install and configure buildbot etc (though we may now have automated that ...) <dread> ah, i see, it's in the cloud <dread> did all that months ago, and don't need to setup the machine any more. <dread> if we do it again though, I'm all for it }}} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #898 | 1294659537000000 | dread | From David Raznick | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #371 | 1294417434000000 | thejimmyg | From #440 we'll also need to "Write and pass comprehensive performance tests" | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #371 | 1294417553000000 | thejimmyg | From #395: At the moment, some pages within CKAN tend to load slowly. We should create a profiling setup in which we can measure response times for complete requests and individual methods calls. This could be used to identify bottlenecks and find an appropriate caching or tuning strategy to improve CKAN performance. NB: We should also agree on a maximum request latency. TODO: Read up on all those QoS tickets to avoid overlapping efforts. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #661 | 1288271266000000 | dread | Friedrich has done the main meat of this in ckanext/blackbox/smoke.py I've extended it to include http basic auth, testing wanted error statuses, checking API calls etc. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #885 | 1293278564000000 | wwaites | Friedrich had some mixed experiences with owslib and some German CSW endpoints: http://pudo.okfnpad.org/geodaten | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #120 | 1254406544000000 | dread | Found two security holes, both only in READ access (not write). Tickets raised for holes: ticket:132 - REST listing packages & groups ticket:133 - search package/group (WUI & REST) Full details of files checked: $ find . -name "*.py" | xargs grep "import ckan.model" ./ckan/presentation.py:import ckan.model as model REMOVED - unused ./ckan/tests/functional/test_group_edit_authz.py:import ckan.model as model - not runtime ./ckan/tests/functional/test_rest.py:import ckan.model as model - not runtime ./ckan/tests/functional/test_package.py:import ckan.model as model - not runtime ./ckan/tests/functional/test_user.py:import ckan.model as model - not runtime ./ckan/tests/functional/test_tag.py:import ckan.model as model - not runtime ./ckan/tests/functional/test_group.py:import ckan.model as model - not runtime ./ckan/tests/functional/test_package_edit_authz.py:import ckan.model as model - not runtime ./ckan/tests/functional/test_revision.py:import ckan.model as model - not runtime ./ckan/tests/functional/test_authz.py:import ckan.model as model - not runtime ./ckan/tests/test_search.py:import ckan.model as model - not runtime ./ckan/tests/models/test_package.py:import ckan.model as model - not runtime ./ckan/tests/models/test_user.py:import ckan.model as model - not runtime ./ckan/tests/models/test_group.py:import ckan.model as model - not runtime ./ckan/tests/models/test_extras.py:import ckan.model as model - not runtime ./ckan/tests/models/test_misc.py:import ckan.model as model - not runtime ./ckan/tests/models/test_authz.py:import ckan.model as model - not runtime ./ckan/tests/forms/test_package.py:import ckan.model as model - not runtime ./ckan/tests/forms/test_group.py:import ckan.model as model - not runtime ./ckan/tests/forms/test_authz.py:import ckan.model as model - not runtime ./ckan/tests/test_converter.py:import ckan.model as model - not runtime ./ckan/tests/getdata/test_data4nr.py:import ckan.model as model - not runtime ./ckan/tests/__init__.py:import ckan.model as model - not runtime ./ckan/tests/test_authz.py:import ckan.model as model - not runtime ./ckan/tests/test_purge_revision.py:import ckan.model as model - not runtime ./ckan/forms/common.py:import ckan.model as model - just validator ./ckan/forms/package.py:import ckan.model as model - validator & package edits ./ckan/forms/group.py:import ckan.model as model - validate & package, wui & rest edits ./ckan/forms/authz.py:import ckan.model as model - package & group wui/rest ./ckan/controllers/rest.py:import ckan.model as model - HOLE ticket:132 - REST listing packages & groups ./ckan/controllers/base.py:import ckan.model as model - paginate filters out deleted ones ./ckan/migration/versions/005_add_authorization_tables.py:import ckan.model as model - not runtime ./ckan/getdata/data4nr.py:import ckan.model as model - not runtime ./ckan/lib/search.py:import ckan.model as model - HOLE ticket:133 - search package/group (WUI & REST) ./ckan/lib/cli.py: import ckan.model as model - not runtime ./ckan/lib/cli.py: import ckan.model as model - not runtime ./ckan/lib/cli.py: import ckan.model as model - not runtime ./ckan/lib/cli.py: import ckan.model as model - not runtime ./ckan/lib/cli.py: import ckan.model as model - not runtime ./ckan/lib/cli.py: import ckan.model as model - not runtime ./ckan/lib/cli.py: import ckan.model as model - not runtime ./ckan/lib/converter.py:import ckan.model - dumper, not runtime ./ckan/lib/base.py:import ckan.model as model - just a remove ./ckan/authz.py:import ckan.model as model - only gets roles ./bin/ckan_spam.py:import ckan.models as model - not runtime ./bin/ckan-correct.py:import ckan.models - not runtime ./test_migrate.py:import ckan.model as model - not runtime | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #734 | 1292587603000000 | dread | Found problems and ticketed: #872 #873 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2707 | 1344504846000000 | toby | Formmating done other requests moved to new tickets as low priority for effort needed Title fixes translations etc http://trac.ckan.org/ticket/2822 Ordering of fields http://trac.ckan.org/ticket/2823 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #928 | 1297115136000000 | rgrp | Forgot to say moved contributors material to http://ckan.org/wiki/Contributors for the time being. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1577 | 1325473564000000 | rgrp | Foreign characters aren't allowed in file names I would image as they will be part of url. worth noting this at some point but don't think this is very high priority and hence deferring out of v1.6. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1447 | 1326296987000000 | nils.toedtmann | For time being, i created a cron script [https://bitbucket.org/okfn/sysadmin/src/default/etc/cron/remove_old_files remove_old_files]. You could just copy it to /etc/cron.daily/, but i recommend to not run it as root: if it's misconfigured, it could wipe the system! So you better copy it to /home/okfn/sbin/ (not /home/okfn/bin/ which often is the sysadmin HG repo), and add it to some unprivileged user's crontab. In most cases, the leftover files are owned by user "www-data", so {{{ $ sudo crontab -e -u www-data }}} and then add something like {{{ 37 4 * * * /home/okfn/sbin/remove_old_files }}} Don't forget to edit the script remove_old_files itself and list the directories you want to be cleaned up. This is already done on s008/eu8 and s019/eu19. dread, do you want to do this for s025/eu25 and see how this goes? ---- Todo nils: verify tomorrow on s019 that it worked properly, e.g. this should show only a few files: {{{ find /var/lib/ckan/nederland/data/sessions/ -type f -amin +$((7*24*60)) -ls }}} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2746 | 1343211146000000 | aron.carroll | For some reason the test servers aren't serving the demo.js file. Assigning to Toby for investigation. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2366 | 1336395238000000 | dread | For my site I need the Authorization Groups sections gone from the authz pages, for v1.7.1, so I've made these check for the presence of any Authorization Groups in commit: [master 8947302]. So it's another useful stepping stone to getting rid of them cleanly. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2251 | 1334144999000000 | toby | For meeting today with Adria * How to add tracking to solr index. Can the data live in tracking_summary? Do we need any extra indexes on table to help indexing? * How will we add order by popularity to search? * Can we add resources too? * Will search be best way to find most popular packages/resources or should we get that pages data from tracking_summary? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2873 | 1345129315000000 | danieljohnlewis | For example see: http://s031.okserver.org:2375/qa/dataset/five_stars Which has: Example dataset as a score of: 4 After examination of the dataset ( http://s031.okserver.org:2375/dataset/example-dataset ) it is made up of a CSV and an HTML file. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1737 | 1337943671000000 | icmurray | For completeness, package_search docs can be viewed at https://github.com/okfn/ckan/blob/master/ckan/logic/action/get.py#L983 @rgrp: are you happy with this ticket now, does it expose all you need? If so, I'll close it... | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #82 | 1256565441000000 | dread | Follow on work in ckan in ticket:173. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1175 | 1307350866000000 | dread | Flavio, can you post here the exception that you get? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1214 | 1310062570000000 | dread | Fixes have gone into the branch for "Tag returned as a JSON object when updating but as a string when requesting. Same with "extras" content." and "Doesn't return rendered_text property on package update.". I've added DELETE to the Access-Control-Allow-Methods on the server, so you can try that now. I've added a failing test for the problem of deleting the last extra. Am discussing with David Raznick how to fix that best. I think the only other ones left to address are: * Relationships use "object" key rather than id or package_id. * underscore in search parameters | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1546 | 1323775424000000 | johnglover | Fixed: https://github.com/okfn/ckan/commit/fbaf4459b7da95c1db0ade165ea09720539c2f80 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1543 | 1324316682000000 | johnglover | Fixed: https://github.com/okfn/ckan/commit/f3b2bb9827b082ee09c74ede85ded21838fc9e80 Needed to refactor fix for 1501 slightly so that the search controller can still use its own url generator. Added new test for search pagination. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1502 | 1322687823000000 | johnglover | Fixed: https://github.com/okfn/ckan/commit/1534d9f16d300f7d5a7aa6bf1a61f120ca7eff3d | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1396 | 1328225696000000 | rgrp | Fixed. https://github.com/okfn/ckan/commit/a22624ffa9b8cac28b78d5ef5f93d5b682558b14 Need to check performance and css possibly as loading lots of datasets could be slow via administrator could be slow. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1178 | 1308045351000000 | rgrp | Fixed. This was due to incorrect config of bucket option. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1559 | 1332242129000000 | rgrp | Fixed. See https://github.com/okfn/ckan/commit/9965818e20f7532b37eb97a3b76c078263eb0dd1 and https://github.com/okfn/ckanext-disqus/commit/ae6bf09a153831e75f9182a4b30921973ff7d080 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1401 | 1327089904000000 | rgrp | Fixed. See https://github.com/okfn/ckan/commit/7088b1fcae685e56a280ec7b396b0f9803a477d9 @dread: not at all clear why we had to change from g.site_url to h.url_for (idea was people could set site_url if the site was not at root ...) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1576 | 1325190496000000 | rgrp | Fixed. Merge cset: https://github.com/okfn/ckan/commit/cb29d62133bb4d060622f2fa752dd281b71a252b | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Note:
See TracReports for help on using and creating reports.
