{23} Trac comments (3729 matches)

Results (1901 - 2000 of 3729)

← 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 →
Ticket Posixtime Author Newvalue
#1106 1303834069000000 rgrp fixed in branch:defect-1106-bugs-with-routing.
#1107 1310134646000000 thejimmyg Child ticket of #954
#1107 1320321170000000 dread Notes to complete this: * ckan/logic/action/get.py:'def package_autocomplete' exists and is the one that should be used * need to ensure all places in the templates and public/scripts use this action directly, not dataset/autocomplete (or its alias package/autocomplete) * ckan/controllers/package.py:'def autocomplete' should be removed. * ckan/config/routes.py dataset/autocomplete should be removed.
#1107 1340632612000000 icmurray autocomplete action has been deprecated in favour of /api/2/util/dataset/autocomplete : https://github.com/okfn/ckan/blob/master/ckan/controllers/package.py#L655
#1108 1310128678000000 thejimmyg Now that we are using ckan.net as thedatahub.org we should go full steam ahead and create a brand new UI to reflect thedatahub.org's new role. See http://ckan.org/2011/07/08/ckan-vision-update/
#1108 1311180204000000 thejimmyg See also comment on #1200 about using the pdeu theme.
#1108 1314878772000000 zephod First, here is a repo with the new DataHub theme: https://bitbucket.org/okfn/ckanext-datahub-theme We plan to integrate this into ckan-core, thereby replacing the current theme.
#1108 1315140879000000 rgrp Completed in https://bitbucket.org/okfn/ckan/changeset/9be1ae232ec3 cset:9be1ae232ec3
#1109 1303862352000000 kindly It is fixed now in 1.3.4.1.
#1109 1303916487000000 dread Close ticket?
#1109 1304698621000000 dread Did we decide that this facility for storing non-strings via the API is a new feature, rather than a bug?
#1109 1305123852000000 dread What's the status?
#1109 1305124697000000 kindly I am happy this is fixed in cset:445fc04333dd.
#1111 1304935139000000 lucychambers CKAN FAQ now complete (Sections 1,2,5,6 - needs uploading onto site): http://ckan.okfnpad.org/FAQ At CKAN Community Meetup it was decided that Sections 3 and 4 would form a separate contributor's manual.
#1112 1304358643000000 dread This is complete in the branch feature-1112-allow-search-all for release 1.3.5
#1113 1304024611000000 kindly cset: 52a3fb230074
#1114 1304085484000000 dread Done in cset:d7bd4b0f89de on default.
#1115 1324054704000000 dread Authorization groups deprecated with the Group refactor #1477
#1116 1304358664000000 dread This is complete in branch defect-1116-boolean-search-options for release-1.3.5
#1117 1304277240000000 nils.toedtmann Oh, it *does* depend against python-pastescript. Ignore. For some reason it was not (completely/correctly) installed with ckanext-datanl on us4, but is a different issue then. Might be due to missing locale "en_GB.utf8" and dpkg-configrue failing.
#1118 1311174062000000 thejimmyg We haven't ever come across this. Perhaps if you could provide an example we can re-open it. Cheers, James
#1119 1304104689000000 rgrp * Merged ckanext-upload into ckanext-storage * Got existing API systemin in ckanext-storage fully working (e.g. api section was broken -- maybe due to changed boto ...) * API now supports GET requests and JSONP where appropriate * Added auth form to API (could be useful for ajax form stuff going forward) * Improved UI of upload including having ckan based file links (using redirects) * Set uploaded-by field on uploaded files * Checked on AWS And GS Also did: * Improved ofs in various ways including change to standard metadata handling and improved testing (see e.g. https://bitbucket.org/okfn/ofs/changeset/36fac29b7775) * Tracked down [https://github.com/okfn/boto/commit/0d9635c8a9785c9b20b44ee93a0679c002961592 bug with boto and metadata updating on GS] and provided patch.
#1121 1307358426000000 dread We don't need to support JSON extras at the moment.
#1122 1304368075000000 dread @pudo I notice you tackling solr indexing of lists/tuples in ckanext-solr cset:ecc2d69df991 so have updated to use this on ckan.net and reindexed, but #1122 still seems an issue - any chance you can take a look?
#1122 1305718009000000 dread This user is now storing his list as a string so is happy. But I'll leave this ticket open, for pudo to decide whether there is an issue with SOLR indexing tuples/lists, as per the previous comment.
#1122 1306662099000000 pudo I don't think there's really a nice way to do this with the current solr indexing library, but I've commented out the list handling code which should really make handling this cleaner.
#1122 1306747706000000 dread Changing this to "won't fix" just to be clear
#1123 1307374691000000 dread I'll change the architecture to "all" as per this article: http://linuxreviews.org/man/deb-control/
#1123 1307790563000000 thejimmyg This isn't necessarily a trivial thing. Let's get the build system for the packages stable before we start changing it all to support alternative architectures. Once the packaging is working well it would be trivial to switch those servers to amd64. I'm sorry, but this isn't worth the investment in manpower at the moment.
#1123 1307795508000000 rgrp You can't get small amazon instances as 64 bit and many people's own machines are 32 bit. How costly is packaging no-arch exactly (it must be more than a simple 'switch' in the build if you are saying this is costly :-) )
#1123 1311863798000000 nils.toedtmann Looks like the packages on apt.okfn.org are now architecture "all" - great. To avoid future confusion, i change this ticket's solution to "fixed".
#1124 1304601771000000 nils.toedtmann I installed the python-ckanext-solr package rom http://apt-alpha.ckan.org/datanl-dev onto us10/DataGM too, and it seems to work just fine, see http://datagm.test.ckan.net/package?q=manchester
#1124 1323168132000000 thejimmyg Solr is now properly supported in the ckan-1.5 repository.
#1124 1323168156000000 thejimmyg Re-open this if it still affects DataGM.
#1126 1304587078000000 dread Fixed in cset:61bb142a6b7c for ckan v1.3.5.
#1127 1304602084000000 wwaites The proposal is not comparable to PEP and DEP because the projects have vastly different requirements. The API stability needs of a programming language or an operating system (e.g. fundamental building blocks that you don't expect to change often or radically) are very different from a web application. The plone one is comparable. The idea itself is a double-edged sword. It will promote stability which is good but can also tend towards rigidity and stagnation which is bad. Each added bit of bureaucracy and process means fewer people will be willing to collaborate or participate in improving the software. Overall, -1.
#1127 1304606094000000 dread Fixed links
#1127 1305128000000000 thejimmyg This is great idea and it has taken on well. +1 from me. We can update/create new CREP policies as needed.
#1127 1305458020000000 rgrp I'm obviously a strong +1 but feel we should stick with the CEP name (we have PEP and DEP and CREP really does sound weird compared to CEP). Also I think we may need to do a bit more to clarify when you just do a good ticket and when one does a CEP (looking at what has occurred already). I think this may be one reason to keep CEPs separate in drafting from trac as will clearly distinguish CEPs from standard tickets.
#1127 1305458489000000 rgrp one other thing: i think the plan is for CEPs to be in restructured text (for easy transfer to docs). This should be mentioned and the template in the wiki should be in ReST.
#1128 1310567803000000 dread Loader in ckanext/bin/loadscotland.py. Done upload to ckan.net: {{{ (pyenv-ckanext)dread@dread-laptop:~/hgroot/ckanext$ scotland_loader -k xyz -H http://ckan.net/api /home/dread/ckan-local/data/scotland-2010-10-18.csv /home/dread/hgroot/pyenv-ckanext/lib/python2.6/site-packages/pylons/templating.py:610: UserWarning: Unbuilt egg for setuptools [unknown version] (/usr/lib/python2.6/dist-packages) Engine = entry_point.load() No handlers could be found for logger "vdm" 2011-07-13 15:29:17,983 INFO [ckanext.importlib.loader] Loading scottish_neighbourhood_statistics 2011-07-13 15:29:20,109 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:22,089 INFO [ckanext.importlib.loader] Loading traveline_scotland 2011-07-13 15:29:24,579 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:27,045 INFO [ckanext.importlib.loader] Loading sqa_qualifications 2011-07-13 15:29:30,869 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:32,561 INFO [ckanext.importlib.loader] Loading scotxed 2011-07-13 15:29:34,017 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:35,286 INFO [ckanext.importlib.loader] Loading iscjis_vocabularies 2011-07-13 15:29:36,592 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:37,782 INFO [ckanext.importlib.loader] Loading ecare_multi-agency_store_data_model_and_vocabulary 2011-07-13 15:29:39,043 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:40,267 INFO [ckanext.importlib.loader] Loading digital_archive_scottish_archive_network 2011-07-13 15:29:41,690 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:43,727 INFO [ckanext.importlib.loader] Loading the_scottish_register_of_tartans 2011-07-13 15:29:45,159 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:46,816 INFO [ckanext.importlib.loader] Loading scotlands_images 2011-07-13 15:29:48,563 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:49,983 INFO [ckanext.importlib.loader] Loading national_library_of_scotland 2011-07-13 15:29:51,709 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:53,122 INFO [ckanext.importlib.loader] Loading scottish_government_statistics 2011-07-13 15:29:55,857 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:29:57,118 INFO [ckanext.importlib.loader] Loading scotbis 2011-07-13 15:29:58,883 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:00,264 INFO [ckanext.importlib.loader] Loading historic_scotland 2011-07-13 15:30:02,086 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:03,609 INFO [ckanext.importlib.loader] Loading scheduled_monuments_in_scotland 2011-07-13 15:30:05,057 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:06,987 INFO [ckanext.importlib.loader] Loading scotlands_house_prices_service_consumer_information 2011-07-13 15:30:08,687 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:09,932 INFO [ckanext.importlib.loader] Loading air_quality_in_scotland 2011-07-13 15:30:11,278 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:12,637 INFO [ckanext.importlib.loader] Loading river_and_coastal_flooding_updates_for_scotland 2011-07-13 15:30:13,940 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:15,637 INFO [ckanext.importlib.loader] Loading public_contracts_scotland 2011-07-13 15:30:17,280 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:18,666 INFO [ckanext.importlib.loader] Loading scottish_government_consultations 2011-07-13 15:30:20,673 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:22,234 INFO [ckanext.importlib.loader] Loading scottish_public_health_observatory 2011-07-13 15:30:24,025 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:26,297 INFO [ckanext.importlib.loader] Loading scotlands_information_landscape 2011-07-13 15:30:28,308 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:29,758 INFO [ckanext.importlib.loader] Loading isd_scotland 2011-07-13 15:30:31,847 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:33,074 INFO [ckanext.importlib.loader] Loading scotlandspeople 2011-07-13 15:30:34,541 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:35,754 INFO [ckanext.importlib.loader] Loading general_register_office_for_scotland_statistics 2011-07-13 15:30:37,273 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:39,330 INFO [ckanext.importlib.loader] Loading baseline_scotland_-_groundwater_chemistry_data 2011-07-13 15:30:40,716 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:41,919 INFO [ckanext.importlib.loader] Loading the_centre_for_the_study_of_retailing_in_scotland_data_sets 2011-07-13 15:30:43,275 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:44,515 INFO [ckanext.importlib.loader] Loading growing_up_in_scotland 2011-07-13 15:30:46,215 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:47,582 INFO [ckanext.importlib.loader] Loading scottish_parliament_msp_allowances 2011-07-13 15:30:49,580 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:51,272 INFO [ckanext.importlib.loader] Loading scottish_government_ministerial_hospitality 2011-07-13 15:30:53,764 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:55,642 INFO [ckanext.importlib.loader] Loading scottish_government_expenditure_over_25000 2011-07-13 15:30:58,012 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:30:59,674 INFO [ckanext.importlib.loader] Loading scottish_local_government_financial_statistics 2011-07-13 15:31:01,869 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:31:03,382 INFO [ckanext.importlib.loader] Loading scottish_government_disclosure_of_high_earners 2011-07-13 15:31:07,086 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:31:11,493 INFO [ckanext.importlib.loader] Loading scotlands_places 2011-07-13 15:31:14,452 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:31:31,743 INFO [ckanext.importlib.loader] Loading scottish_national_heritage_facts 2011-07-13 15:31:36,223 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:31:37,830 INFO [ckanext.importlib.loader] Loading scottish_natural_spaces 2011-07-13 15:31:40,245 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:31:41,764 INFO [ckanext.importlib.loader] Loading scotland_national_address_gazetteer 2011-07-13 15:31:43,810 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:31:45,069 INFO [ckanext.importlib.loader] Loading scottish_environment_protection_agency 2011-07-13 15:31:47,975 INFO [ckanext.importlib.loader] ...creating package 2011-07-13 15:31:49,525 INFO [ckanext.importlib.loader] Loading traffic_scotland 2011-07-13 15:31:52,396 INFO [ckanext.importlib.loader] ...creating package }}}
#1128 1310568028000000 dread Note, I added a title column and made one or two minor corrections that I spotted - see attached CSV for the result.
#1129 1305116419000000 thejimmyg Great proposal, here's my thinking: * The current VDM model (where we have things like a single package table which logs changes to a package_revision table) is very well tested and works well and we should only consider a radical change with good reason * Although we are making a move from *model objects* being the central components to the *logic layer functions* being the central components, there is no need yet to have the database structures we store (for revisioning or otherwise) start to look like the dictized representation. The logic layer itself is designed to do that mapping. * Since we do have this logic layer though, there is less need for "magic" SQLAlchemy objects to support dicts and lists (eg the stateful objects used to make a list of tags on a package behave like a list), a few simple (and easily debuggable) queries in the logic layer would work better. In the longer term we may want to look at serialising more dictized-looking objects and that may lend itself to a more dictized changeset model or even a more dictized storage system (eg no-SQL) but for the time being we are not at that point. I recommend the following: * We continue to use the current default branch of VDM (not the changeset branch) * We continue to treat the package table (and other non-revision tables) as the most recent revision (even though the *active* revision displayed in CKAN might well be in the revision tables because more recent changes haven't been moderated yet) * We slowly stop using stateful lists and dicts in CKAN because we have move control with explicit queries in the logic layer Sound good?
#1129 1305139980000000 rgrp It seems like this ticket is getting a bit confused with ticket:1076 (changes re vdm) :-) Some general conclusions relevant to both: * General feeling is not to change to changeset model * I'm personally +0 on changeset model * I would note main benefit is simplicity and orthogonality of changesets to core system. I also think cost of change is small. * That said we can simplify current model as well. * There seems some misunderstanding: change to have logic layer has almost nothing to do with being able to remove main stateful stuff in vdm. To be able to remove most of stateful stuff in vdm requires us to make some other changes (re foreign keys from revision objects to continuity) * There are other simplifications we should make to vdm before embarking on this (e.g. move to SessionExtension from MapperExtension). This is easy as that work has been done in changeset branch and can be backported. What I don't see in this CEP as yet is any discussion of the complexities around pending stuff (e.g. do we allow multiple pending, do new edits get shown current or latest pending). I'll try and comment more on this but a lot of this was in original vdm discussion last summer.
#1129 1305209601000000 rgrp Replying to [comment:2 rgrp]: > It seems like this ticket is getting a bit confused with ticket:1076 (changes re vdm) :-) I of course ;0 meant ticket:1077
#1129 1305212467000000 kindly > * There seems some misunderstanding: change to have logic layer has almost nothing to do with being able to remove main stateful stuff in vdm. To be able to remove most of stateful stuff in vdm requires us to make some other changes (re foreign keys from revision objects to continuity) The logic layer does not automatically help out, however it makes our life easier if we want to handle state ourselves. For example take package tags, if we remove the stateful_m2m properties and just use normal sqlalchemy relations. We will still want statefullness (i.e active, pending, deleted) on the package_tags table. We should update those on the table ourselves in the logic layer. > * There are other simplifications we should make to vdm before embarking on this (e.g. move to SessionExtension from MapperExtension). This is easy as that work has been done in changeset branch and can be backported. I agree but event thought the MapperExtension way is not great, it is very well field tested.
#1129 1325268100000000 rgrp More UI work remains in #1141
#1130 1340633514000000 icmurray Unassign for triage.
#1131 1305537897000000 dread Fixed on branch release-v1.3.5 cset:3461ed6bab0c
#1132 1307352675000000 dread Yes, these are apparently search-related and so are skipped when testing under sqlite. See README.rst for more info. Compare with this: {{{ $ nosetests --ckan --with-pylons=test-core.ini ckan/tests/functional/test_authz.py ............................................ ---------------------------------------------------------------------- Ran 44 tests in 69.287s OK }}}
#1133 1324057072000000 dread Works for me - change occurs successfully. {{{ (pyenv-ckan)dread@dread-laptop:~/gitroot/ckan$ paster rights add russianfan admin warandpeace /home/dread/gitroot/pyenv-ckan/lib/python2.6/site-packages/pylons/templating.py:610: UserWarning: Unbuilt egg for setuptools [unknown version] (/usr/lib/python2.6/dist-packages) Engine = entry_point.load() User russianfan -> is admin on -> Package warandpeace (pyenv-ckan)dread@dread-laptop:~/gitroot/ckan$ paster rights /home/dread/gitroot/pyenv-ckan/lib/python2.6/site-packages/pylons/templating.py:610: UserWarning: Unbuilt egg for setuptools [unknown version] (/usr/lib/python2.6/dist-packages) Engine = entry_point.load() 22 results User visitor -> is editor on -> Package annakarenina User logged_in -> is editor on -> Package annakarenina User testsysadmin -> is admin on -> System system User logged_in -> is editor on -> System system User visitor -> is reader on -> Group david User annafan -> is admin on -> Package annakarenina AuthorizationGroup anauthzgroup -> is reader on -> Package warandpeace User visitor -> is editor on -> Package warandpeace AuthorizationGroup anauthzgroup -> is editor on -> Group david User logged_in -> is editor on -> Package warandpeace User logged_in -> is reader on -> Group david User visitor -> is reader on -> Group roger User logged_in -> is reader on -> Group roger User russianfan -> is admin on -> Package warandpeace User visitor -> is admin on -> Package warandpeace AuthorizationGroup anotherauthzgroup -> is editor on -> AuthorizationGroup anauthzgroup User russianfan -> is admin on -> Group roger User russianfan -> is admin on -> Group david AuthorizationGroup anauthzgroup -> is admin on -> AuthorizationGroup anotherauthzgroup User visitor -> is anon_editor on -> System system User visitor -> is admin on -> Group roger AuthorizationGroup anauthzgroup -> is editor on -> System system }}} Tests would be nice, but it currently works, is only for admins and is therefore not as high priority as other other tests, such as front-end js.
#1134 1305125228000000 thejimmyg Great stuff. Yes, I agree with all this, but as initial thoughts I suggest that for phase 1, the info() method returns one more key called "form_config_interface" which takes a string representing a the type of interface for the config field on the form. If the key is missing it is treated as "None". The two possible values are: None - the field is not present and must always be stored as NULL Text - a single text field will be provided on the interface Whatever the interface, the value built will always be stored in the config column of the table. We then also provide a "get_schema()" method that returns a schema cablable of parsing the data submitted by the form and storing it as a single key named "config". ckanext-inspire will then add other functions to the schema for URL etc so that regardless of what the harvest plugin does, the key fields ckanext-harvest are always there. This then allows: * Customiseable config interfaces * Customisable valiation * A consistent place to store config in the database Sound good?
#1134 1305623531000000 sebbacon Agreed with James that we should consider *from the start* how to provide for a nice UI -- my worry is if we start out with people editing raw JSON we'll end up keeping that for years :)
#1135 1305210134000000 rgrp Work on this is largely complete in changeset branch: https://bitbucket.org/okfn/vdm/src/changeset/ Overview docs: https://bitbucket.org/okfn/vdm/src/changeset/doc/changeset.rst
#1135 1340632267000000 icmurray Re-assigned to kindly
#1136 1305210925000000 rgrp Most of this change has been implemented in the changeset branch [1] but could be backported reasonably easily to main. [1]: https://bitbucket.org/okfn/vdm/src/changeset/
#1136 1305216218000000 kindly This would be nice but it is not necessary. The current mapper implementation may be nice but not is battle tested..
#1136 1305217553000000 kindly Replying to [comment:2 kindly]: > This would be nice but it is not necessary. The current mapper implementation may be nice but not is battle tested.. I meant the mapper extension "IS" battle tested.
#1136 1340632980000000 icmurray Re-assigned to @kindly.
#1137 1305211672000000 rgrp Most of this change has been implemented in the changeset branch [1] but could be backported reasonably easily to main. [1]: https://bitbucket.org/okfn/vdm/src/changeset/
#1137 1305217449000000 kindly I am not sure this needs to be done. I think we should keep the continuity object always in the table, even if it is deleted. The querying should be done through the logic layer so the deleted state should not be an issue. The clients should be entirely state aware. The only thing that needs to be done is to remove all statefullness of relations. These are the only things that are complicated. This would make vdm just a simple copy on write mechanism, with the client controlling the state.
#1138 1316965357000000 rgrp Cannot see this issue any more on default (as deployed on e.g. test).
#1140 1312491320000000 kindly fixed cset:987da68ea4f6 The package group table needed to trigger a reindex of the package.
#1140 1312537257000000 dread Broken since CKAN v1.1. I have added this fix to v1.4.2 and v1.4.3 releases.
#1141 1308828928000000 dread I've merged this into default in preparation for release 1.4.2, but there may be a couple of useful additions that could be done on this or another branch/ticket. Getting a package at a specified revision is really useful. This can be used: * links in the package history (#103) (while in there, the 'moderated' flag should be shown in the package history too if not already) * should also be added to the package API (as a replacement for the AJAX call?) The history_ajax call looks like an almost duplicate of the Package Revision API. Can these be merged and put just in the API?
#1141 1318862183000000 dread What's the status of this now? The ckanext-moderatededits README still says you need a branch of ckan. Can you say which version of CKAN is required instead? I say we close this if ajax call is now in a better place in the API. (I expect it is now, with the logic layer).
#1141 1318862773000000 johnglover Well, it runs on datacatalogs.org with the current default so CKAN 1.5, but it needs quite a bit of updating to work with all of the new 1.5 UI changes, and as it has been a low priority for quite a while I haven't scheduled time to work on it. Really I would need to spend a few days on it to tidy it up for standard 1.5.
#1141 1325352507000000 dread Closing, with remaining tidy up work for newer CKAN versions split off into new ticket: #1604
#1142 1305880901000000 sebbacon The guys at Wikimedia also produced this spreadsheet including notable users and features of other software. Could be useful.: https://spreadsheets2.google.com/ccc?authkey=CLWu1cgP&hl=en&key=tYiwc9kUVLLlALOQEzFqujw&hl=en&authkey=CLWu1cgP#gid=0
#1142 1305881244000000 lucychambers Replying to [ticket:1142 rgrp]: > Previous super ticket (from 3m ago): http://trac.ckan.org/ticket/927 > > * CKAN 1-page overview (for enterprise and for data hackers) > * Administrator's Guide (including install) > * Extensions Guide > * Separate CKAN.net info from Software Documentation (?) > > == Minor Items == > * Include guide on how to configure local filesystem storage, instead of Google storage. (http://lists.okfn.org/pipermail/ckan-discuss/2011-May/001235.html)
#1142 1308326118000000 annapowellsmith List of new docs tasks agreed by APS and RGRP: * Tutorial on the wiki (long-term): Integration with Wordpress and Drupal and Javascript * Overview of domain model both for average users and for devs * 2-page glossy brochure * Vision for CKAN http://notebook.okfn.org/2011/04/27/data-hubs-data-management-systems-and-ckan/ * Comparison : use http://wiki.ckan.net/Related_Software as starting point - build on wiki - Socrata and OGDI * Renaming: separation between ckan.org and ckan.net * More documentation in the code ... * Config options generated from code or move this to WUI ... See also proposed overhaul at http://wiki.ckan.net/Documentation_Plans
#1142 1313775665000000 rgrp This was completed 4 weeks ago and can now be marked as complete. See http://docs.ckan.org/ and http://wiki.ckan.net/ and http://ckan.org/
#1143 1307024681000000 dread All done in ckanext-stats cset:bc54790cacd4 (apart from "fix x axis to start at first revision" which was not a problem after all)
#1148 1305969925000000 kindly complete cset:96a43c9d8bd7
#1149 1305971672000000 kindly cset:b1634d405066
#1149 1306090663000000 kindly Failed dgu tests due to as_dict not working on deleted objects. fixed by cset:c9b9cf513e44
#1150 1311178163000000 thejimmyg Hi John, Can you please check that the new webstore+preview works correctly with this one too please? Cheers, James
#1150 1311774141000000 johnglover Unicode data should be fine in the new datapreview code, for example: http://test.ckan.net/package/afghanistan-election-data
#1151 1330908188000000 rgrp Not sure we will be able to fix this easily as we need a dataproxy for general case. Think it better to focus on the case where data in DataStore and we use Recline. As such closing as wontfix for the present.
#1152 1307352165000000 amercader Bumping to ckan-v1.5-sprint-3 and updating the CC email addresses so people actually get any updates.
#1154 1314120497000000 johnglover Fixed in core (branch feature-1275-solr-search).
#1154 1314177477000000 dread Looking at the changeset, it now does a connection check at start-up of CKAN. If it fails then CKAN runs but doesn't show any packages, apart from via the Tags interface, and the search page and widget on the front-page are removed. (BTW Is this a useful thing to do? Without search, perhaps CKAN should simply not start?) I believe the problem pudo is referring to is temporary periods when the SOLR server is slow/overloaded/down. During this time pudo and I get exception emails saying connection failed for request. e.g. {{{ Module ckan.controllers.home:29 in index << query = query_for(model.Package) query.run(query='*:*', facet_by=g.facets, limit=0, offset=0, username=c.user) c.facets = query.facets c.fields = [] >> limit=0, offset=0, username=c.user) Module ckan.lib.search.common:115 in run << self.query = QueryParser(query, terms, fields) self.query.validate() self._run() self._format_results() return {'results': self.results, 'count': self.count} >> self._run() Module ckanext.solr.backend:63 in _run << # this wrapping will be caught further up in the WUI. log.exception(e) raise SearchError(e) finally: conn.close() >> raise SearchError(e) SearchError: [Errno 111] Connection refused }}} (recently from nl.ckan.net)
#1154 1314178325000000 johnglover > Looking at the changeset, it now does a connection check at start-up of CKAN. If it fails then CKAN runs but doesn't show any > packages, apart from via the Tags interface, and the search page and widget on the front-page are removed. (BTW Is this a > useful thing to do? Without search, perhaps CKAN should simply not start?) When discussing the ticket we agreed that we should allow ckan to start without search. It is possible that people simply want to list/view information without searching for it and we decided not to prevent them from doing so. > I believe the problem pudo is referring to is temporary periods when the SOLR server is slow/overloaded/down. During this > time pudo and I get exception emails saying connection failed for request. Ok, other options then: * if an error occurs during search, return 0 results * if an error occurs during search, redirect to a generic 'search is unavailable' page * if an error occurs during search, retry the query a certain number of times before giving up and doing one of the above. What do you suggest?
#1154 1314179290000000 dread Search requests - option one seems like a bad user experience, and option three seems too complicated. I should do option 2, explain it may well be a temporary problem, and let the user press reload, as per any normal web request. A more difficult problem though is what to do about a failed search indexing. Because of the try/except catch we've had until the patch yesterday, we get no exception emails for this, but I'm worried that this has caused packages to not be indexed, and hence are essentially lost in CKAN (unless someone realises there is a problem and does a reindex at some point). BTW I added paster command 'search-index check' for looking at this problem, but I don't think it was ever added to solr. Is it easy to add get_all_entity_ids for the purpose of checking for this problem?
#1154 1314191496000000 johnglover Ok I'll implement option 2 for search query issues. Yes it is easy to get a list of IDs from Solr so I'll have a look at that command and make sure that it works. I think a case could be made for not allowing a package to be added if it cannot be indexed however, and making the user simply retry the request.
#1154 1314192324000000 dread Yes, forcing a retry may be possible, although I guess on these occasions you'd be looking at rolling back the creation of the package object too, which is not really in the spirit of the notifications system at present. This is why we had the queue for solr for occasions like this - is it here no longer?
#1154 1314192753000000 johnglover Not in core, there is a queue feature in the extension still which I could add to core. I think though it was a case of using either the queue or synchronous search, I don't think it would automatically fall back to the queue if there was an indexing problem, but we could look at that. Was there not talk of changing the queue system recently though? There are also no tests for the Solr queue so that would take a bit of work as well.
#1154 1314193633000000 johnglover RE search queries failing: Just having a look through the controller code and the package search controller and the search api both perform option 2 already, so I'll just update the home controller.
#1154 1314195684000000 dread Ok, I chatted to David about this and we feel it should go like this: notification to search indexer fails and raises an exception, this gets passed through the notification system back up to the code that creates the package, and you do a roll back. Does this tally with what you mentioned in the last comment or is what's there better? Thanks for the info on the queue and ckanext-solr. It sounds like this isn't deployed successfully anywhere (or does pudo know if it is?). In this case we should be clear that it is not what it says it is: "This extension adds support for package search using Apache Solr to CKAN" and maybe simply mark it as deprecated. Pudo what do you think?
#1154 1314196683000000 johnglover > Ok, I chatted to David about this and we feel it should go like this: notification to search indexer fails and raises an exception, > this gets passed through the notification system back up to the code that creates the package, and you do a roll back. Does this > tally with what you mentioned in the last comment or is what's there better? This sounds good to me, definitely better than what is currently there. > Thanks for the info on the queue and ckanext-solr. It sounds like this isn't deployed successfully anywhere (or does pudo > know if it is?). I'm not actually sure if the queue works or not, but if it does I'll have to write some tests for it before we could add it to core. A working queue system would be useful generally however, as there are other things that it would be useful for (such as QA), but yes, going forward the Solr extension should really be deprecated.
#1154 1314287519000000 johnglover All attempts to index packages should now throw a SearchIndexError if Solr is unavailable and the create/update will be rolled back. These errors are caught in the main package controller (redirecting to an error page) and the API controller. I have added tests for both controllers. Also updated the paster 'search-index check' command to work with solr.
#1155 1306773174000000 pudo Now implemented based on CSV export available on the catalogue page: * http://data.london.gov.uk/catalogue
#1156 1306855111000000 pudo Both are now implemented but may need further work to adhere to developing conventions wrt to location etc.
#1159 1307615133000000 pudo Rudimentary RDFa has been added with seeAlso to API.
#1161 1306408026000000 amercader Duplicate of #1157
#1164 1308647224000000 amercader Done. See it in action here: http://publicdata.eu/map
#1165 1306413953000000 nils.toedtmann this is similar, but not equal to #1096
#1165 1307444733000000 nils.toedtmann See also http://ckan.okfnpad.org/multisite
#1166 1306847326000000 thejimmyg Also, rather than INSPIRE=true we should probably have information about the harvesting mechanism and the harvest object type? This also relates to our use of the kind field on the resource.
#1166 1317381932000000 amercader Fixed on d5bee3de9957
← 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 →
Note: See TracReports for help on using and creating reports.