{23} Trac comments (3729 matches)

Results (1601 - 1700 of 3729)

← 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 →
Ticket Posixtime Author Newvalue
#790 1289815658000000 johnbywater Moved from sprint 1.3.3
#793 1289815658000000 johnbywater Moved from sprint 1.3.3
#794 1289815658000000 johnbywater Moved from sprint 1.3.3
#798 1289815658000000 johnbywater Moved from sprint 1.3.3
#800 1289483682000000 johnbywater Referencing ticket #799 has changed sprint.
#801 1289483616000000 johnbywater Referencing ticket #728 has changed sprint.
#801 1289483682000000 johnbywater Referencing ticket #799 has changed sprint.
#191 1323706210000000 johnglover working in current master and 1.5.1
#191 1324030875000000 johnglover Thanks for this David. This one in particular was causing me some grief: https://github.com/okfn/ckan/commit/f98a4b2a5f5013fa4aed475bd8b3237bb7847fcc Good spot. All looks good to me.
#318 1311770683000000 johnglover The QA code should identify invalid URLs. Resources with invalid urls will have an 'openness_score' of 0 and an 'openness_score_reason' of 'Invalid url scheme' or 'invalid URL'.
#816 1312295371000000 johnglover I have implemented the resource autocomplete so am closing this for now. I agree that more information on the resource would be a good addition. It seems a bit of a different issue to just adding autocomplete though, with potentially far more changes to the codebase, so maybe it would be better discussed in a separate ticket/CREP. However, any new fields would presumably want to either be constrained or have an autocomplete and so can reuse the work done on this feature. Cheers, John
#888 1311773103000000 johnglover Dataproxy / Dataapi now deprecated in favour of the combination of new QA archive / process commands and the webstore. Changes in relation to Dataproxy / Dataapi: * Currently only supports CSV files, but plans to add support for excel and google docs spreadsheets soon. * Uses David Raznick's CSV parser instead of Brewery for parsing, handles messy CSV data better. Changes in relation to old QA functionality: * decoupled archiving (downloading) and QA process * added a new 'process' command which parses downloaded files and adds them to a local webstore Closing for now, any improvements/feature requests should be in tickets relating to either the QA functionality or the webstore.
#891 1320143424000000 johnglover Almost finished (see http://github.com/okfn/ckanext-archiver). Still to address: - check headers to see if hash / cache / max-age / expires indicates that the resource does not need to be downloaded. - add cache url to resource
#891 1320149841000000 johnglover Added cache_url and cache_last_updated to resources after archiving. Not checking for hash value in headers. This process will generally only run when a new resource is added or someone updates a URL, so we don't expect to be regularly downloading the same resource. We will need something along these lines if this is running as a regular cron job, but in that case the logic will be added to the cron job itself (probably a paster command).
#892 1324402480000000 johnglover https://github.com/okfn/ckan/compare/bff538b%5E...d700680 Merged with master and deployed on test.ckan.org
#1062 1311773731000000 johnglover The file is in Turtle format (serialisation format for RDF, which we currently don't archive and have no preview code for. The new preview code will not even create a preview button for this file type. Moving to the backlog for now, support for RDF and related formats should be added along with support for excel docs and google spreadsheets.
#1141 1318862773000000 johnglover Well, it runs on datacatalogs.org with the current default so CKAN 1.5, but it needs quite a bit of updating to work with all of the new 1.5 UI changes, and as it has been a low priority for quite a while I haven't scheduled time to work on it. Really I would need to spend a few days on it to tidy it up for standard 1.5.
#1150 1311774141000000 johnglover Unicode data should be fine in the new datapreview code, for example: http://test.ckan.net/package/afghanistan-election-data
#1154 1314120497000000 johnglover Fixed in core (branch feature-1275-solr-search).
#1154 1314178325000000 johnglover > Looking at the changeset, it now does a connection check at start-up of CKAN. If it fails then CKAN runs but doesn't show any > packages, apart from via the Tags interface, and the search page and widget on the front-page are removed. (BTW Is this a > useful thing to do? Without search, perhaps CKAN should simply not start?) When discussing the ticket we agreed that we should allow ckan to start without search. It is possible that people simply want to list/view information without searching for it and we decided not to prevent them from doing so. > I believe the problem pudo is referring to is temporary periods when the SOLR server is slow/overloaded/down. During this > time pudo and I get exception emails saying connection failed for request. Ok, other options then: * if an error occurs during search, return 0 results * if an error occurs during search, redirect to a generic 'search is unavailable' page * if an error occurs during search, retry the query a certain number of times before giving up and doing one of the above. What do you suggest?
#1154 1314191496000000 johnglover Ok I'll implement option 2 for search query issues. Yes it is easy to get a list of IDs from Solr so I'll have a look at that command and make sure that it works. I think a case could be made for not allowing a package to be added if it cannot be indexed however, and making the user simply retry the request.
#1154 1314192753000000 johnglover Not in core, there is a queue feature in the extension still which I could add to core. I think though it was a case of using either the queue or synchronous search, I don't think it would automatically fall back to the queue if there was an indexing problem, but we could look at that. Was there not talk of changing the queue system recently though? There are also no tests for the Solr queue so that would take a bit of work as well.
#1154 1314193633000000 johnglover RE search queries failing: Just having a look through the controller code and the package search controller and the search api both perform option 2 already, so I'll just update the home controller.
#1154 1314196683000000 johnglover > Ok, I chatted to David about this and we feel it should go like this: notification to search indexer fails and raises an exception, > this gets passed through the notification system back up to the code that creates the package, and you do a roll back. Does this > tally with what you mentioned in the last comment or is what's there better? This sounds good to me, definitely better than what is currently there. > Thanks for the info on the queue and ckanext-solr. It sounds like this isn't deployed successfully anywhere (or does pudo > know if it is?). I'm not actually sure if the queue works or not, but if it does I'll have to write some tests for it before we could add it to core. A working queue system would be useful generally however, as there are other things that it would be useful for (such as QA), but yes, going forward the Solr extension should really be deprecated.
#1154 1314287519000000 johnglover All attempts to index packages should now throw a SearchIndexError if Solr is unavailable and the create/update will be rolled back. These errors are caught in the main package controller (redirecting to an error page) and the API controller. I have added tests for both controllers. Also updated the paster 'search-index check' command to work with solr.
#1183 1311771069000000 johnglover Fixed in new datapreview code (currently running on the test.ckan.net), which now gets the preview data from a webstore instead of the dataproxy API. It also only now shows a preview button for resources that were successfully archived and saved in the webstore.
#1275 1314110016000000 johnglover Completed in branch feature-1275-solr-search. Up to date with current default but not yet merged into default. John
#1276 1314886010000000 johnglover Added to branch feature-1275-solr-search Don't seem to be able to apply weighting to fields at index time using solrpy (see open issue: http://code.google.com/p/solrpy/issues/detail?id=6). So, switched to using the newer Solr edismax query parser and have applied weights to fields at search time. Current weights are as follows: name: 4 title: 4 group: 2 tag: 2 text (everything else): 1
#1277 1314868204000000 johnglover Done in branch feature-1275-solr-search for all package searches (which use Solr). Resource and Tag searches still use the old ckan query parser.
#1293 1314701675000000 johnglover == Changes == * All templates * Routing: /package/ -> /dataset/ (including API) * Tests: h.url_for(controller='dataset') rather than package ... * One test to check redirect works? * Docs == Leave == * Stay with Package in code e.g. stay with 'PackageController' * also leave stuff like pkg = .... (gradual migration) * Leave extensions == Notes == * Package -> Dataset * Data Package -> Dataset * respect capitalization
#1293 1314905799000000 johnglover Branch: feature-1293-rename-package-to-dataset Routing and test editing should be finished (all tests passings). Have not added test for redirect yet.
#1363 1317656645000000 johnglover table: task_status id entity_id entity_type task_type key value date_updated state unique(entity_id, task_type, key) example keys: last_task_id
#1371 1317833379000000 johnglover Done, in branch feature-1371-task-status-logic-layer
#1392 1319625243000000 johnglover Now on s054.okserver.org
#1393 1318508765000000 johnglover Yes, there are a few of these problems with SQLite which is why the tests are being skipped currently. I'm not exactly sure of the reason, it seems that it sometimes drops tables from memory, but we haven't found a solution. Myself and David Raznick were going to spend some time looking at it but haven't had a chance yet.
#1400 1328612209000000 johnglover I haven't written any docs for this. I'm also not sure what the status is, Ross was doing some work on it recently so he is probably the man to ask.
#1400 1329746131000000 johnglover Will document and update wiki this sprint then ping Mark about a blog post.
#1400 1330524586000000 johnglover Async task docs are at: http://wiki.ckan.org/Writing_asynchronous_tasks Archiver readme at: http://github.com/okfn/ckanext-archiver Emailing Mark RE blog post.
#1400 1331543941000000 johnglover I thought that we had some sort of consensus that the official docs were aimed more at system admins and people using the CKAN WUI/API, with the wiki aimed more at developers? It would be easy to move of course, but there is no real "developers" section in the official docs that I can see, and it's even called "CKAN's Administration Guide".
#1400 1331544816000000 johnglover I'll have a look thanks. I'm +1 on using main docs instead (if we move everything), I was just trying to keep things consistent for now.
#1436 1320230896000000 johnglover This also applies to CKAN core
#1436 1320243278000000 johnglover Fixed in branch feature-1371-task-status-logic-layer, will merge with default after 1.5 release later today.
#1449 1321465348000000 johnglover Basic change done in branch feature-1449-resource-listing. Currently showing: * resource name (clickable) or (none) * resource description * resource format * resource last modified (if exists)
#1450 1321875986000000 johnglover Still working on this, moving to new sprint
#1451 1321876014000000 johnglover Haven't looked at this yet, moving to new sprint
#1451 1322567112000000 johnglover This was taken from the UX pad. I'm looking at it now to make sure that it's working and that we can use it on the new dataset view page. Is it really a good idea to move it to core though? There was no reason given on the pad so I'm not sure why this should be moved to core?
#1451 1323165781000000 johnglover Scripts and templates updated for CKAN 1.5.1 Waiting for Tom to finish dataset view and resource view updates, then will discuss best place to put the download stats on the page.
#1451 1324288407000000 johnglover Need to add analytics javascript to new resource view page, will do under ticket #1519.
#1451 1324401792000000 johnglover This ticket again brought up the issue of a need for inversion of control for writing to CKAN templates (main templates should really ask extensions for data or provide hooks rather than having extensions overwrite sections using genshi/jquery). Have decided to defer this issue for now, as it will be looked at when looking at making our whole extension system more robust early next year. Related point: * We should provide a dashboard area (again via interfaces/hooks) that groups together report data from related extensions. So stats and googleanalytics info should be available from a similar area / url. This is also deferred until the extension overhaul/refactor. Other outstanding issues: * Location / style of download count can be improved * Download count should be shown on resource pages These should be looked at by whoever takes over frontend design.
#1455 1322491506000000 johnglover Fixed - commit: https://github.com/okfn/ckan/commit/05b675a4314ad269c6e6a095d57e3f2a21e771eb Note: - Includes a small change to Solr schema file. - Search index will need to be rebuilt for changes to take effect
#1463 1321874259000000 johnglover This works with the new Celery feature that will be in 1.5.1 (which should be released in this sprint). So, will not update this old version of QA for 1.5, people should use the new version (on okfn Github) after 1.5.1 is released.
#1468 1321875777000000 johnglover New ticket, moving over to new sprint
#1468 1322495417000000 johnglover Done - commit: https://github.com/okfn/ckan/commit/7789e85c973c9e085f623486bced6be14f25678f rebuild can now take an optional package name/id (single package to be consistent with other paster commands, not a list of packages)
#1481 1322062707000000 johnglover Fixed in new dataset UX changes (branch feature-1450-dataset-view), still has to be merged with master.
#1483 1323279151000000 johnglover Fixed. Updated logic layer: https://github.com/okfn/ckan/commit/520959ca6f83586d33cf7d4015d625a776ee46a1 Updated tasks (qa, archiver, webstorer): https://github.com/okfn/ckanext-qa/commit/78cbf5c6d16225e1580aee23fdf884f88d2475f5 https://github.com/okfn/ckanext-qa/commit/8553348942adad0babd4bdd91820c1d75a1784a7 https://github.com/okfn/ckanext-archiver/commit/8ed419280bd7f2e66aeb4f17f3f546d941fe11fb https://github.com/okfn/ckanext-webstorer/commit/d16fb090533a716c856e59838f79bbaa0ea42e76
#1501 1322684411000000 johnglover Fixed, commit: https://github.com/okfn/ckan/commit/255352a1c2f403aa5d25b461ca60dbc272d90ea4
#1502 1322687823000000 johnglover Fixed: https://github.com/okfn/ckan/commit/1534d9f16d300f7d5a7aa6bf1a61f120ca7eff3d
#1519 1324297045000000 johnglover Didn't have time to look at this in previous sprint, moving to current sprint.
#1543 1324296991000000 johnglover New ticket, moving to current sprint
#1543 1324316682000000 johnglover Fixed: https://github.com/okfn/ckan/commit/f3b2bb9827b082ee09c74ede85ded21838fc9e80 Needed to refactor fix for 1501 slightly so that the search controller can still use its own url generator. Added new test for search pagination.
#1546 1323775424000000 johnglover Fixed: https://github.com/okfn/ckan/commit/fbaf4459b7da95c1db0ade165ea09720539c2f80
#1582 1324489033000000 johnglover celery and ckanext-qa both deployed to thedatahub. ckanext-archiver also installed as the link checker is used by ckanext-qa, but the plugin is not activated. QA scan of all datasets (CRON job) currently scheduled once a week (Sunday, 4:30am)
#1582 1326110801000000 johnglover blog post added: http://ckan.org/2012/01/09/qa-on-thedatahub/
#1587 1339433286000000 johnglover Fixed on ckanext-qa branch release-v1.7.1
#1594 1338203215000000 johnglover We're not using super tickets any more so closing this, most of the smaller tickets are still open however.
#1622 1326127683000000 johnglover Done. http://dgu-os.okfn.org/qa/dataset/five_stars Full QA scan scheduled to run every Sunday night at 04:00 UK time using crontab of user ckandguos.
#1629 1326215334000000 johnglover anon_editor and editor were given the create-package role, I had previously removed it via paster roles deny [anon_]editor create-package.
#1692 1338204395000000 johnglover Image URL attribute added to groups (http://trac.ckan.org/ticket/2275). Won't add a similar attribute to dataset at present. Another option is to display the group image on dataset page.
#1699 1327425070000000 johnglover CKAN 1.5.1 with EC Portal extension has been deployed to test server. Full setup docs are in the ckanext-ecportal extension.
#1700 1327425247000000 johnglover Done, see ecportal_setup.rst in ckanext-ecportal extension.
#1703 1327425195000000 johnglover First draft complete (see ckanext-ecportal extension), ideally would need sample data from additional publishers for testing.
#1703 1328528711000000 johnglover Moving to backlog for now. Work based on current schema (from original proof-of-concept site and later meetings) complete, waiting for final metadata schema to be confirmed.
#1703 1329918613000000 johnglover Some metadata model info clarified at meeting today, will update ckanext-ecportal accordingly.
#1703 1330956340000000 johnglover Nearly done, need to write converter to rename tags to keywords, waiting for Toby's API work to be finished.
#1703 1331142926000000 johnglover Marking this stage as complete. Keywords can be added/updated via WUI and API, but are still labelled 'tags' when called via package_show API call, but this will be addressed in a separate ticket.
#1704 1327425354000000 johnglover Importer made and added to ckanext-ecportal/scripts. Still has to be run on the test server with Eurostat JSON datasets.
#1707 1328024253000000 johnglover Tom fixed this as part of #1583
#1720 1328528809000000 johnglover Moving over to current sprint. Basic converters added, still have to write tests.
#1721 1329133348000000 johnglover Done: https://github.com/okfn/ckan/commit/bf3c85fbf9c72c0e8ac27d2079c20e07b6c7c668
#1724 1328634433000000 johnglover I updated tag_list: https://github.com/okfn/ckan/commit/335535b646adf983d1d19bf19613712099db1f5e
#1729 1328537269000000 johnglover This is essentially a duplicate of #1720. We do not really need helper functions for this (as it's trivial to get the default schema and add 1 extra key), but I have added a converter for tag string (with vocab) to tag and vice versa.
#1730 1328537387000000 johnglover The function for this is the converter function for tags with vocabs. Will integrate with the JQuery chosen plugin.
#1809 1330528828000000 johnglover Caught exceptions in QA instead, error message stored as openness_score_reason
#1810 1329918479000000 johnglover Deferred for now, moving to backlog.
#1810 1335874745000000 johnglover Not using eurovoc tags.
#1811 1329918517000000 johnglover Deferred for now, moving to backlog
#1811 1335874765000000 johnglover Not using eurovoc tags.
#1812 1335874864000000 johnglover Ross has fixed this via group authentication.
#2206 1331046486000000 johnglover Close enough for now. Still have to restyle language drop-down to match template, but will fix when updating the list of languages. Also still to decide on menu styles/locations (for CKAN login and main menu, and ODP main menu).
#2212 1331719868000000 johnglover Added as a new paster command in ckanext-ecportal (run 'paster ecportal' for a list of commands and arguments).
#2212 1331720191000000 johnglover Deployed on test server, where it imported 4087 datasets. A small number of datasets were not created as they failed CKAN validation - most of which had strange values such as 9999 and 0 for date fields (some also didn't have unique names).
#2260 1333373978000000 johnglover Still some work to do on this, template needs to be finalised.
#2260 1333554069000000 johnglover Finished for this iteration, waiting for design work to be completed before making further changes.
#2275 1334159668000000 johnglover Done in branch feature-2275-group-logo, still need to decide on how to style.
#2346 1338205553000000 johnglover Will create a separate ticket for 'organization' to 'publisher' renaming, the rest are done.
#2464 1338910189000000 johnglover Fixed for now, but had to keep group type as 'organization' as the string is hardcoded into the publisher auth profile in 1.7.1.
#2470 1338284041000000 johnglover Search results validated against schema. Added a class implementing IPackageController, in after_search results are replaced with package dicts got from package_show. A bit heavy-handed, but works for now. Maybe we should just call package_show in the package_search logic function in core?
#2487 1338892521000000 johnglover https://github.com/okfn/ckanext-ecportal/commit/958208cfec1dfddb792b9b0ce0ac835fb1e2872a
#2488 1339412538000000 johnglover Works for me on 1.7.1. Perhaps floapps was using older groups (of type 'group' instead of 'organization') and/or didn't add a user to the group?
#2509 1339419867000000 johnglover https://github.com/okfn/ckan/pull/20
← 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 →
Note: See TracReports for help on using and creating reports.