{23} Trac comments (3729 matches)
Results (3601 - 3700 of 3729)
| Ticket | Posixtime | Author | Newvalue | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| #2759 | 1343651082000000 | aron.carroll | Fixed in 16bb516 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2765 | 1343817858000000 | aron.carroll | Cool, closed as of 99b8c4f. I fixed the error summary display and highlighted the fields in red to show they were incorrect. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2767 | 1343645517000000 | aron.carroll | Closed as of 25d7499. All form.select() calls now require a tuple/list in the form. {{{ options = ( {"value": "opt1-value", "text": "Option One Text"}, {"value": "opt2-value", "text": "Option Two Text"} ) }} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2774 | 1344008312000000 | aron.carroll | I'm happy to just make ajax requests to public/base/i18n/<locale>.js, however this will use the language set in the <html lang> attribute. So if this file doesn't exist then we'll get a 404 and show the english. If we want something more sophisticated, determine an appropriate fallback etc, then I think the API is the way to go. Or we could get CKAN to determine the appropriate language file and include it in the page source... | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2776 | 1343824375000000 | aron.carroll | Done in f24fcd9. This will require the demo site to be updated. This line needs to be added to the config.ini to get the CKAN logo back. {{{ ckan.site_logo = /base/images/ckan-logo.png }}} For other local sites remove the logo and the site title and description will be used. Currently the design doesn't handle a long description so beware. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2778 | 1343841130000000 | aron.carroll | Tidied up in a55523175a95f2fc5c0ddddf9fa9579aa7111e39 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2779 | 1343818300000000 | aron.carroll | Toby is there a flag on the dataset object that tells me if it has been deleted? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2779 | 1343903455000000 | aron.carroll | Done in d17358f | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2781 | 1343907075000000 | aron.carroll | What do you mean by "hiding behind that black rectangle"? Just tried it locally with the text "Welcome to the CKAN demo Try out standard CKAN functionality in a sandbox environment. Search for datasets directly from the homepage or by navigating to the Datasets search page where you can facet by tags, groups and format." and it renders okay… | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2781 | 1344510922000000 | aron.carroll | Fixed as of 7c86e31 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2791 | 1343906884000000 | aron.carroll | I've added no-wrap as of 1db2037. Not sure what we should do about short titles/long tag lines though. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2792 | 1343901501000000 | aron.carroll | Fixed in 1a83a77 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2794 | 1343903064000000 | aron.carroll | Done in 11f3b6297c | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2797 | 1344250973000000 | aron.carroll | Just a classname issue. Fixed in f927598 and documented | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2801 | 1344526719000000 | aron.carroll | Not sure it is. I'm just implementing a toggle on description so it works the same way as on the datahub. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2801 | 1344531016000000 | aron.carroll | Now able to display the full text of related items as of e83df46 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2802 | 1344269446000000 | aron.carroll | Cool, this is in as of 8b72d1f8e9ba | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2803 | 1344256274000000 | aron.carroll | Done in 3b2427e | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2804 | 1344269865000000 | aron.carroll | Done in 38f824b | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2807 | 1344270190000000 | aron.carroll | Fixed in 9a6d0be | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2808 | 1344944258000000 | aron.carroll | Close in 97d92ba | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2809 | 1344347814000000 | aron.carroll | Fixed in 84a4123 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2824 | 1344532640000000 | aron.carroll | All fixed | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2834 | 1344856692000000 | aron.carroll | Done in 092d257 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2836 | 1344852009000000 | aron.carroll | Yeah you can pass an html block into a macro like so: https://github.com/okfn/ckan/blob/2375-demo-theme-development/ckan/templates/package/snippets/resource_form.html#L44-48 What do you think? If it's useful I'll document it. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2837 | 1344847463000000 | aron.carroll | Checkboxes created with {{{ form.checkbox() }}} should be styled correctly. Got a link to an example? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2837 | 1344851271000000 | aron.carroll | Fixed alignment issues in Firefox as of a6fa5a0. Can't do anything about visual display of checkboxes as they're controlled by the vendor. We could replace them with images, if this is going to be required then please create a new ticket for the next phase. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2840 | 1344852538000000 | aron.carroll | Fixed in f7bc1c9. This can be flipped back by adding a class of {{{.tagline-right}}} to the hgoup element. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2841 | 1344857415000000 | aron.carroll | Fixed in a317fb9 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1214 | 1310053225000000 | aron | Replying to [comment:2 dread]: > Aron, > I've fixed the second item (on my branch), so you can DELETE a package without Content-Length. But I'm not sure what you mean here: "Should be 405 Method Not Allowed?" DELETE should be allowed. Or is that a response you're getting? > David If I send a HEAD request to the server eg. {{{ curl -i -XHEAD http://test.ckan.net:80/api/2/rest/package/ec9cb930-d15f-441f-a1e1-36f4d5df19bf }}} I get the following in the Access-Control-Allow-Methods header {{{ Access-Control-Allow-Methods: POST, PUT, GET, OPTIONS }}} I've just realised that the "Access-Control" headers are for cross origin browser requests so I think you can ignore my comments about the correct response being 405 Method Not Allowed but perhaps DELETE should be added to the Access-Control-Allow-Methods list? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1214 | 1310054305000000 | aron | Replying to [comment:3 dread]: > "Tag returned as a JSON object when updating but as a string when requesting." > I'm not sure what you mean. > > http://test.ckan.net/api/2/rest/package/reference-example > gives tags: {{{"tags": ["country-uk", "example-package"],}}} I'm getting back objects for each tag in the response body from a PUT request. For example "country-uk" would be something like. {{{ { "id" : "600dc72e-6127-4704-b801-bee00474ec0c", "name" : "country-uk", "revision_timestamp" : "2011-07-06T09:09:21.578034", "state" : "active" } }}} Hopefully this gist <https://gist.github.com/97447d0b28bf52f3e06b> will illustrate the issue. > Is it this - returning package names, rather than ids? Here is certainly a bug I'll fix. > http://test.ckan.net/api/2/rest/tag/country-uk That was the fifth point on the list. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1214 | 1310054863000000 | aron | Replying to [comment:5 dread]: > Deleting an extra using 'null' works for me: Hmm, it works for me too when there are other keys remaining in the "extras" object. However I can't seem to delete the last one. Take the following example. {{{ curl -i -XPUT -d'{"extras": {"Tester": null}}' -H"X-CKAN-Type: application/json" http://test.ckan.net:80/api/2/rest/package/ec9cb930-d15f-441f-a1e1-36f4d5df19bf }}} Gives: {{{ { "author" : "", "author_email" : "", "extras" : [ { "id" : "88cc50ca-9e29-4499-a542-09d364f5f64f", "key" : "Tester", "package_id" : "ec9cb930-d15f-441f-a1e1-36f4d5df19bf", "revision_id" : "e5c3ca9c-1dae-4a86-837f-b8c19ac31964", "revision_timestamp" : "2011-07-07T16:01:05.696025", "state" : "active", "value" : "\"Test Value\"" } ], "groups" : [ ], "id" : "ec9cb930-d15f-441f-a1e1-36f4d5df19bf", "license_id" : "", "maintainer" : "aron", "maintainer_email" : "", "name" : "my-test-package", "notes" : "Heading\r\n===\r\nThis _is_ some text", "relationships_as_object" : [ ], "relationships_as_subject" : [ ], "resources" : [ ], "revision_id" : "688d33fb-5629-4ab3-9f59-a649fc7caa00", "revision_timestamp" : "2011-07-06T10:37:50.182894", "state" : "active", "tags" : [ { "id" : "600dc72e-6127-4704-b801-bee00474ec0c", "name" : "test-tag", "revision_timestamp" : "2011-07-06T09:09:21.578034", "state" : "active" } ], "title" : "My Test Package", "url" : "", "version" : "" } }}} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1214 | 1310055169000000 | aron | Another edge case that popped up while verifying the above issues. A PUT request without a body throws a 500 Internal Sever Error {{{ curl -i -XPUT -H"X-CKAN-API-KEY: tester" -H"Content-Type: application/json" http://test.ckan.net:80/api/2/rest/package/ec9cb930-d15f-441f-a1e1-36f4d5df19bf }}} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1214 | 1310057942000000 | aron | One more for today. Please do let me know if I should be filing these as separate tickets? For some reason performing a search query for packages with an underscore "_ " as a query string key fails to return any results. {{{ curl "http://test.ckan.net/api/2/search/package?q=osm" }}} Gives me 4 results. {{{ curl "http://test.ckan.net/api/2/search/package?q=osm&_=1310056826904" }}} Gives me none. The underscore is generally used by JavaScript libraries as a way of bypassing the browser cache when making JSONP calls. It's easily worked around but is odd none the less. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1214 | 1310396399000000 | aron | The groups property in the package resource is always empty (or is for all packages I have viewed). The following group lists one package. {{{ http://test.ckan.net/api/2/rest/group/0ac963e7-ba29-49bc-83c8-98f8c1991649 }}} But when viewing the package the groups array is empty. {{{ http://test.ckan.net/api/2/rest/package/758c26d4-5949-4347-8b4d-023374146d94 }}} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #204 | 1285081954000000 | anonymous | [http://www.grattage.pro Casinos pro] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #204 | 1285082169000000 | anonymous | [http://www.casinotop10.fr Top10 des casinos sur internet] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #321 | 1291831399000000 | anonymous | This has now been superseded with this proposal: #787 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #349 | 1277820679000000 | anonymous | Mostly done, but issue regarding departments still outstanding: can the association between packages, groups, and departments be placed elsewhere? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #371 | 1294676093000000 | anonymous | Mainly handled in http://knowledgeforge.net/okfn/tasks/ticket/564 now. Close here? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #407 | 1291149724000000 | anonymous | Related bug on ScraperWiki tracker: https://bitbucket.org/ScraperWiki/scraperwiki/issue/42/finish-ckan-integration | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #441 | 1282724585000000 | anonymous | Comment from pudo: CKAN should have a read-only maintenance mode with a nice little banner on all pages, appropriate REST messages etc. Bonus points if this is triggered via an environment variable and thus can be triggered by the surrounding apache. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #546 | 1286896167000000 | anonymous | [http://www.pokers.li Jeux de poker en ligne] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #546 | 1286896450000000 | anonymous | [http://www.bonus-pokers.eu Poker en ligne] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #546 | 1286896814000000 | anonymous | [http://www.salle-de-poker-legal.com Jouer au poker en ligne] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #698 | 1293472613000000 | anonymous | Data proxy documentation: http://democracyfarm.org/dataproxy/api.html (included in sources) Updated ('s' as in structured) data proxy app: http://sdataproxy.appspot.com | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #823 | 1290506116000000 | anonymous | Fixed in cset:3845a501ed5f | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #868 | 1293401808000000 | anonymous | Attached are the timings I have for the tests after I upgraded to 0.57 and after a few simple test tweaks. They do not include setup and teardown time at the class level as they are not assignable to individual tests. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #870 | 1294914243000000 | anonymous | Merged into default in cset:54ae110094be | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #876 | 1293188088000000 | anonymous | Thanks for your feedback, very useful. I don't really agree with the people in the linked discussion who say it's pointless testing against a different database from production. The goal here is to make it easy enough for people to run as many tests as possible that they actually do so. Even 15 minutes is too long in that case. With sqlite we can get it in at under 5 minutes. I would also like to identify the longest running tests (which I would characterise as "functional" or "integration" tests and make them run as a separate suite, and then encourage a culture of writing true unit tests before functional tests, so that running unit tests can happen in 1 minute and be part of the regular development cycle. That's no replacement for also running *all* tests periodically, and also running tests under postgres, which we can continue to do on the continuous integration server. Longer term I agree that it would be better to run local tests against postgres too, but that will I think involve refactoring many of the tests. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #876 | 1293218714000000 | anonymous | I agree with all your points about testing apart form using sqlite, especially splitting out the functional tests and continuous integration. > Longer term I agree that it would be better to run local tests against postgres too, but that will I think involve refactoring many of the tests. Well there are two options 1. refactor the tests 2. refactor the code to use sqlite and postress It is a value judgment as to which is more complicated. I personally think 2 is more complicated but may be wrong on that. The real danger with 2 is that you are needlessly adding complication to production code, with 1 you are only changing the tests. Upgrading to sqlalchemy 0.5+ should happen first regardless. You will need upto date documentation. There is another option too. Put the postgres data directory on tempfs/ramfs and turn off durability [http://www.postgresql.org/docs/9.0/static/non-durability.html here]. We would need a way to db init before the tests where run (or) at boot). This may be the best of both worlds. Anyway Happy xmas!! | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #926 | 1298541597000000 | anonymous | Goals: We want the interface for updating an object to be loosely coupled to the method for updating it. We might update a Package from: - HTML forms - a REST API (using JSON) - a CLI (potentially using command line arguments, YaML, XML or ini files) Right now, data is validated using a form framework, even if we're not using forms. Data is written to the object as part of the forms framework (using the "sync()" method), making the process hard to customise and hard to discover. Instead, there should be a standard chain for: - deserialising untyped data (such as that received from an HTTP POST or parsed from a YaML file) into valid data - returning structured errors suitable for displaying to the user - saving the validated, deserialised data Ideally, it would look something like: schema = MySchemaDefinition() raw_data = open("raw.csv", "r").read() structured_data = to_python(raw_data, schema) try: validated = validate(python_data) myobject.update_from_dict(validated) return "Updated OK" except ValidationError, e: return "Error: %s" % e.to_dict() The inverse would be something like: structured_data = myobject.render_to_dict() raw_data.write(to_csv(structured_data, schema) print "Wrote CSV %s" % to_logformat(serialized_data, schema) The question of how to generate and display forms should be completely decoupled from this. It should be easy to write forms by hand, which means it should be simple to flatten the serialized data to key, value pairs, and match up any validation errors to each key. Optionally, a form widget generation framework is a nice-to-have, but not essential, as it is expected that, given enough time, the majority of forms will require manual coding to accomodate edge conditions. A form widget generation framework should be reasonably complete if it's worth trying at all, which means it should support things like: - nested fields (at least repeating, multi-value fieldsets) - widgets for dates and file uploads - internationalisation ...but note I'd settle for *no* widget generation Components of a serialisation / validation framework: - a simple, obvious way to define a schema - a lightweight validation implementation - simple interface for validators - easy to match validation errors to data structure items Overall, I'd like to see: - loose coupling, no framework dependencies - maximal test coverage - extensive documentation with readily available examples ## Findings I looked at flatland, formencode, FormAlchemy, formish, WTForms, Django, web2py, deform/colander, formconvert and web.py - **web2py** just helps build HTML from python, so isn't what I'm after at all - **web.py** has rudimentary validation which is only aimed at HTML forms and is hence tightly coupled with them. - **Django**'s forms are again tightly coupled to HTML forms (and their generation) - **FormAlchemy** similarly couples validation to forms, and is focussed on inferring a schema from a data model SQLAlchemy. - **WTForms** again focuses on Form generation and don't make itx easy to deserialise arbitrary data This leaves us with Flatland, Formencode, Formish, Colander/Peppercorn/Deform, and FormConvert. Having reviewed all of these, I rejected Formencode on the basis of its patchy documentation and relatively low unit test coverage. I also found it mixed concerns a bit much for my taste. Formish felt similarly sparsely documented. Of the remainder, I'd be happy using any of them, but opted for Colander in the end as it has the most exhaustive documentation and unit tests and has been used in production for a long time. FormConvert has a nice design but is a bit of a moving target at the moment -- worth revisiting in the future. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #982 | 1297518386000000 | anonymous | Well we could transfer all the dependencies and version numbers to a config file for the fabfile, but we don't achieve much. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #982 | 1297850732000000 | anonymous | This is now rolled into #963. Marking as duplicate. People can get the pip from a branch over HTTP like this: https://bitbucket.org/okfn/ckan/src/<branch-name>/path/to/file/you/want | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #998 | 1298371191000000 | anonymous | I am happy to get rid of paster db create altogether as a compromise? Or add a depreciation warning to it? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1142 | 1308326118000000 | annapowellsmith | List of new docs tasks agreed by APS and RGRP: * Tutorial on the wiki (long-term): Integration with Wordpress and Drupal and Javascript * Overview of domain model both for average users and for devs * 2-page glossy brochure * Vision for CKAN http://notebook.okfn.org/2011/04/27/data-hubs-data-management-systems-and-ckan/ * Comparison : use http://wiki.ckan.net/Related_Software as starting point - build on wiki - Socrata and OGDI * Renaming: separation between ckan.org and ckan.net * More documentation in the code ... * Config options generated from code or move this to WUI ... See also proposed overhaul at http://wiki.ckan.net/Documentation_Plans | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1245 | 1311864530000000 | annapowellsmith | Would be nice to list some extensions somewhere on this site (features page perhaps). Along the lines of: http://docs.ckan.org/extensions.html#finding-extensions It wasn't until I wrote this list that I realised just how many extensions there were, and what it said about the CKAN community that there were so many... So I think it would be a good sales point. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #736 | 1304963598000000 | amercader | See https://bitbucket.org/okfn/ckanext-harvest/changeset/a7b6d8c0dde7 https://bitbucket.org/okfn/ckanext-harvest/changeset/40de12fada74 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #906 | 1328782087000000 | amercader | Yes, I don't think there will be any problem, and we won't need to create a new version of the schema as the change is backwards compatible. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #985 | 1306407617000000 | amercader | Moving this to the PDEU (publicdata.eu) project | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #985 | 1306408134000000 | amercader | Duplicate of #1155 and #1156 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1030 | 1300103984000000 | amercader | This includes: * Take out all references to harvestsource, harvestingjob and harvesteddocument in the rest API * Move the harvesting bits of ckan/lib/cli.py into ckanext-harvest. * Move ckan/controller/harvesting.py and cknan/model/harvesting.py to ckanext-harvest as well * Update ckanext-csw to be able to find the code it needs in the new place. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1030 | 1303117978000000 | amercader | All the points described in this ticket have been completed. Harvesting work is being done mainly under #1037 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1152 | 1307352165000000 | amercader | Bumping to ckan-v1.5-sprint-3 and updating the CC email addresses so people actually get any updates. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1161 | 1306408026000000 | amercader | Duplicate of #1157 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1164 | 1308647224000000 | amercader | Done. See it in action here: http://publicdata.eu/map | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1166 | 1317381932000000 | amercader | Fixed on d5bee3de9957 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1229 | 1312191530000000 | amercader | Done, except for Revision related stuff, which probably overlaps with work being done by David Raznick. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1338 | 1316951545000000 | amercader | Reopening, because another change is needed to support custom schemas | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1338 | 1317117738000000 | amercader | Fixed in eca1edce3a0f | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1345 | 1343215489000000 | amercader | See https://github.com/okfn/ckan/pull/73 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1354 | 1316970374000000 | amercader | There is no way to maintain backwards compatibility, so let's forget about it for now | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1375 | 1318260008000000 | amercader | Fixed in 888ed50c098d, using Session.flush() rather than the proposed one. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1381 | 1319021092000000 | amercader | Now implemented on 6227142b0460 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1430 | 1320339962000000 | amercader | I've been digging more on this one. To reproduce it, you just have to edit the same dataset in both sites (production and testing). Just after editing the dataset, the search index will get mixed site_ids. I checked the jetty logs (see attached files) and just after editing a dataset there are two POST requests to update the index. The request logs don't show the requests params so it's hard to tell what the second call does (it probably is the commit): https://bitbucket.org/okfn/ckan/src/97e1e90d66d7/ckan/lib/search/index.py#cl-144 In any case, it's clear that the problem may be related with the datasets in the two cores sharing the same id. We are currently using the dataset id as uniqueKey in SOLR, i.e in our schema.xml, we are defining: {{{ <uniqueKey>id</uniqueKey> }}} According to the SOLR docs: "If a document is added that contains the same value for this field as an existing document, the old document will be deleted." http://wiki.apache.org/solr/SchemaXml#The_Unique_Key_Field I would expect the uniqueKey not to be mixed between cores, but it looks like it happens otherwise. Maybe we should generate a solr_id specific to each document for each site, as described here: http://wiki.apache.org/solr/UniqueKey#UUID_techniques (Note that apart from the testing/production site use case, at some point sites involved in harvesting could also end up with datasets with the same id.) Again, I'm not a SOLR expert, so the problem could be a completely different one! | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1430 | 1320431138000000 | amercader | Right, more news on this front. I've tested a patch which uses a hash of the dataset id and site_id to produce a unique id, and then configured the iati solr cores to use index_id as uniqueKey: https://bitbucket.org/okfn/ckan/changeset/855f5a452f60 Unfortunately, that did not solve the issue. Again, updating the same dataset in both apps messes things up. In this case, documents don't get replaced, but duplicated, so the new uniqueKey is working. I am more inclined to think that this is caused by a misconfiguration in the SOLR instance in s046. This is the file where the two cores are configured: {{{ <solr persistent="true" sharedLib="lib"> <cores adminPath="/admin/cores"> <core name="testing.iatiregistry.org" instanceDir="testing.iatiregistry.org"> <property name="dataDir" value="/usr/share/solr/testing.iatiregistry.org/data" /> </core> <core name="iatiregistry.org" instanceDir="iatiregistry.org"> <property name="dataDir" value="/usr/share/solr/iatiregistry.org/data" /> </core> </cores> </solr> }}} Following this paths: /usr/share/solr/iatiregistry.org symlinks to /etc/solr/iatiregistry.org, /etc/solr/iatiregistry.org/data is empty (as well as the testing equivalent). On the other hand, looking at the admin interface and at some errors that I got it seems that the data folder that both cores are using is /var/lib/solr/data/index Maybe that's the problem? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1430 | 1320660144000000 | amercader | No, I didn't know it. Is this supposed to be the correct setting or should each core have its own data dir? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1430 | 1320670062000000 | amercader | As rgrp mentioned, we commented out the <dataDir> directive in the solrconfig.xml files and rebooted. That made the cores use the data dir they were supposed to (the one in solr.xml) and from the tests I made looks like it finally fixed the issue. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1462 | 1326284357000000 | amercader | Closing as this has been fixed and deployed. @thejimmyg Not sure if there are still issues regarding packaging. Feel free to create a specific ticket for this if we need to work on it. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1469 | 1329760150000000 | amercader | This is mostly done (current form is http://i.imgur.com/zmfc5.png). Still some tests missing and a little bit of cleanup and documentation required. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1490 | 1322489155000000 | amercader | Fixed in c23821b | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1498 | 1323165876000000 | amercader | This is now done. See [1] for the deployment documentation and conventions. CKAN checks the remote schema version on startup to see if it's compatible. See commits starting from 50285ef6. The migration work needed after releasing 1.5.1 is described in #1516. [1] https://github.com/okfn/ckan/blob/feature-1498-multiple-schemas/doc/solr-setup.rst | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1516 | 1323360020000000 | amercader | Closing as the main Solr servers are ready to support different Solr schemas. I.e they have two different cores: * http://<server>/solr/ckan-schema-1.2 * http://<server>/solr/ckan-schema-1.3 to which CKAN instances can point to. The CKAN instances that have not been updated (the ones under s004) are pointing to a Solr core with an old version of the schema, so they can wait until upgraded to 1.5.1 to update the solr_url property and rebuild the index. Data.gov.uk can be dealt with during the next deployment. It's not clear which Solr server are using the rest of the instances, but they can be updated as necessary when they upgrade their CKAN instance. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1540 | 1326060385000000 | amercader | Fixed in b80da7a4fe | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1590 | 1326711608000000 | amercader | Added in 658f76e4a0 and deployed to the production site. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1592 | 1326304321000000 | amercader | Fixed on 1e649a62fcf | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1616 | 1326710037000000 | amercader | Sounds like a good option. I will move it to ckan-v1.6 anyway because it may need some work than the expected. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1616 | 1332327635000000 | amercader | Fixed in 10cfd168413 Also new flags and fixes for other commands were added: * Add -o option to only reindex datasets not already indexed * Add -i option to ignore exceptions when rebuilding * Add -r option to just refresh the index (not clearing it first) * Fix show command to show the index stored for a dataset * Add support for clearing the index of just one dataset | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1640 | 1326710888000000 | amercader | Depends on #1655 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1640 | 1327340939000000 | amercader | Done, see http://publicdata.eu/package?extras_eu_country=RS | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1641 | 1326376777000000 | amercader | Fixed on 77fa6483c32 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1677 | 1326807655000000 | amercader | See #1678 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1701 | 1327423221000000 | amercader | Done. See 668292 - 78cc11 Time spent: 1.5 days | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1701 | 1330085360000000 | amercader | Sorry, this ticket referred to the EC Portal project, the changesets are from ckanext-ecportal. This hasn't gone into core yet, which is ticket #906 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1731 | 1330942924000000 | amercader | The underlying auth layer is done, still there isn't UI integration (list of publishers in index page, publisher field in form...). Needs to be moved to the next sprint. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1734 | 1332167315000000 | amercader | All sub tickets have been implemented, so closing it now. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1801 | 1343144718000000 | amercader | Closing as both old and new theme now show password reset links | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #1816 | 1331302835000000 | amercader | +1 to a new, more flexible extension. But in the meantime I've just spent a couple of hours making it work with latest CKAN, which was easier than expected, so we can deploy it with the new version of PDEU. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2232 | 1332331660000000 | amercader | Fixed in 6b4c6c9 This was due to some characters not being supported by XML. We now strip them before indexing. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| #2252 | 1333108787000000 | amercader | There is a new ticket for the same schema as forms issue: #2268 The exception was fixed on [https://github.com/okfn/ckanext-inspire/commit/0dff1f6 0dff1f6] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Note:
See TracReports for help on using and creating reports.
