{23} Trac comments (3729 matches)

Results (1901 - 2000 of 3729)

← 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 →
Ticket Posixtime Author Newvalue
#987 1311177705000000 thejimmyg This has largely been implemented now for publicdata.eu. There is a CREP #1134 outstanding to take the harvesting to the next level so marking this one as duplicate for now.
#1062 1311178145000000 thejimmyg Hi John, Can you please check that the new webstore+preview works correctly with this one please? Cheers, James
#1150 1311178163000000 thejimmyg Hi John, Can you please check that the new webstore+preview works correctly with this one too please? Cheers, James
#225 1311178276000000 thejimmyg This ticket is more than 6 months old so closing it in line with our new ticketing policy.
#1167 1311178516000000 thejimmyg I've now written a 20 page guide on using CKAN on Amazon EC2. I haven't explicitly created an AMI, but I don't think we should do that anyway for a few reasons: * it is fiddly * there is no advantage to the user as CKAN installation is so easy and so well documented * we don't want to have to maintain the image going forward * we might not want to promote a non-open approach The guide will form part of Anna PS's work on the overall documentation but you can also see the original version here: https://bitbucket.org/okfn/ckan-debs-public/src/489a5ecf369f/ec2/
#543 1311178918000000 thejimmyg Consolidation of caching has been moved in ticket #995.
#537 1311178929000000 thejimmyg Consolidation of caching has been moved in ticket #995.
#995 1311179009000000 thejimmyg See also previous tickets: * #537 Caching and Performance improvement * #543 Investigate partial page caching and edge-side includes
#103 1311179429000000 thejimmyg David, I'm in the middle of a ticket refactor. Please don't open tickets I've just closed ;) This will be taken forward as part of #1233
#1204 1311179980000000 thejimmyg Sorry, is this a CKAN issue or a datacatalogs.org one? If it is datacatalogs.org do you mean a catalog rather than a package? If so, can you give an example of the sort of change you've made? Marking as invalid until we get more information.
#1108 1311180204000000 thejimmyg See also comment on #1200 about using the pdeu theme.
#1200 1311180218000000 thejimmyg This is a duplicate of #1108. Let's have the discussion there please.
#830 1311180263000000 thejimmyg We can now implement themes via extensions or change them via config options. See https://bitbucket.org/okfn/ckanext-exampletheme/ for an example.
#103 1311180850000000 dread Ah, you should have mentioned the new ticket! ;-)
#990 1311180850000000 thejimmyg This now works.
#78 1311181280000000 thejimmyg This ticket is now more than 6 months old so closing in line with our new ticketing policy.
#143 1311181336000000 thejimmyg Baze is looking into this and also into most followed packages.
#165 1311181391000000 thejimmyg This is more than 6 months old so closing.
#1183 1311181719000000 thejimmyg Hi John, Can you have a look at this one too please? Cheers, James
#1096 1311182262000000 thejimmyg Milestone ckan-v-future deleted
#249 1311182450000000 thejimmyg This is over 6 months old so closing in line with our ticketing policy. Also, I shouldn't think we really want such a complicated search anyway.
#449 1311182945000000 thejimmyg This ticket is now more than 6 months old so marking as invalid in line with our ticketing policy.
#659 1311183031000000 thejimmyg Since there is already an implementation and already exists as a ticket in the OKFN tasks, I'm marking this as closed. If there is a reason this is still open, please add it to a more appropriate ticket, such as the API version 3 one.
#660 1311183115000000 thejimmyg This ticket is more than 6 months old so closing it in line with our ticketing policy.
#1234 1311183328000000 dread Fixed in cset:1ed3fde56a61 for 1.4.3.
#820 1311325226000000 dread This has been fixed
#1207 1311325343000000 dread This has been fixed.
#1195 1311332049000000 dread Also this one: http://ckan.net/storage/f/file/
#1236 1311351476000000 dread Working on it on branch enh-1236-view-package-revision
#1239 1311442163000000 rgrp Fixed in cset:adf254b7c507 (see https://bitbucket.org/okfn/ckan/changeset/adf254b7c507)
#1208 1311525306000000 rgrp Propose this ticket can be closed as webstore is now functional and being used. Can open new tickets for specific improvements / adaptations as the need arises.
#1238 1311583876000000 kindly I cannot reproduce this, to me it looks like it does get the correct revision. If you for example look at the package a millisecond before i.e http://ckan.net/package/osm%402010-11-30%2000%3A21%3A49.627829 the tags are no longer there that were added in that revision.
#1002 1311675754000000 dread Can you also delete the "paster changes" in ckan/lib/cli with this removal?
#1002 1311676199000000 dread Reopening this, since it has not been pushed or merged in afaict.
#1236 1311699843000000 dread This is merged to default (for rel 1.2.5) but isn't brilliant yet because of bug #1238.
#1238 1311759784000000 dread Apologies, I have a problem instead with the way I identify the revision in #1236.
#1236 1311759816000000 dread The bug is in fact in the way I work out what revision I'm displaying - it just looks at the PackageRevision.
#318 1311770683000000 johnglover The QA code should identify invalid URLs. Resources with invalid urls will have an 'openness_score' of 0 and an 'openness_score_reason' of 'Invalid url scheme' or 'invalid URL'.
#1183 1311771069000000 johnglover Fixed in new datapreview code (currently running on the test.ckan.net), which now gets the preview data from a webstore instead of the dataproxy API. It also only now shows a preview button for resources that were successfully archived and saved in the webstore.
#888 1311773103000000 johnglover Dataproxy / Dataapi now deprecated in favour of the combination of new QA archive / process commands and the webstore. Changes in relation to Dataproxy / Dataapi: * Currently only supports CSV files, but plans to add support for excel and google docs spreadsheets soon. * Uses David Raznick's CSV parser instead of Brewery for parsing, handles messy CSV data better. Changes in relation to old QA functionality: * decoupled archiving (downloading) and QA process * added a new 'process' command which parses downloaded files and adds them to a local webstore Closing for now, any improvements/feature requests should be in tickets relating to either the QA functionality or the webstore.
#1062 1311773731000000 johnglover The file is in Turtle format (serialisation format for RDF, which we currently don't archive and have no preview code for. The new preview code will not even create a preview button for this file type. Moving to the backlog for now, support for RDF and related formats should be added along with support for excel docs and google spreadsheets.
#1150 1311774141000000 johnglover Unicode data should be fine in the new datapreview code, for example: http://test.ckan.net/package/afghanistan-election-data
#1195 1311841763000000 rgrp Fixed in https://bitbucket.org/okfn/ckanext-storage/changeset/49734285f1ef
#1246 1311863583000000 dread We have an unresolved issue with the display of the 'open data' icon that showed up when we upgraded ckan.net to the latest beta last week, but I didn't realise the search page http://ckan.net/package was showing all packages as open (the revision page and package pages seem fine). Last week when I upgraded from 1.4.1 to 1.4.2beta, there was an exception thrown when loading the home page. The exception was in doing the isopen call. It was solved by making some code changes directly on the ckan.net server to make it an attribute query instead of function call. It was strange because this behaviour didn't occur on my machine and all the tests passed. Here are the manual patches on ckan.net (the application.js change is unrelated, by someone else): {{{ diff -r adf254b7c507 ckan/public/scripts/application.js --- a/ckan/public/scripts/application.js Sat Jul 23 18:16:15 2011 +0100 +++ b/ckan/public/scripts/application.js Thu Jul 28 14:10:23 2011 +0000 @@ -2,6 +2,7 @@ $(document).ready(function () { CKAN.Utils.setupUserAutocomplete($('input.autocomplete-user')); CKAN.Utils.setupAuthzGroupAutocomplete($('input.autocomplete-authzgroup')); + // $('.resource-url').parent( }); }(jQuery)); diff -r adf254b7c507 ckan/templates/_util.html --- a/ckan/templates/_util.html Sat Jul 23 18:16:15 2011 +0100 +++ b/ckan/templates/_util.html Thu Jul 28 14:10:23 2011 +0000 @@ -43,7 +43,7 @@ standard package listing --> <ul py:def="package_list(packages)" class="packages"> <li py:for="package in packages" - class="${'fullyopen' if (package.isopen() and package.resources) else None}"> + class="${'fullyopen' if (package.isopen and package.resources) else None}"> <div class="header"> <span class="title"> ${h.link_to(package.title or package.name, h.url_for(controller='package', action='read', id=package.name))} @@ -60,7 +60,7 @@ </ul> </py:if> <ul class="openness"> - <py:if test="package.isopen()"> + <py:if test="package.isopen"> <li> <a href="http://opendefinition.org/okd/" title="This package satisfies the Open Definition."> <img src="http://assets.okfn.org/images/ok_buttons/od_80x15_blue.png" alt="[Open Data]" /> @@ -72,7 +72,7 @@ </a> </li> </py:if> - <py:if test="not package.isopen()"> + <py:if test="not package.isopen"> <li> <span class="closed"> ${h.icon('lock')} Not Openly Licensed diff -r adf254b7c507 ckan/templates/home/index.html --- a/ckan/templates/home/index.html Sat Jul 23 18:16:15 2011 +0100 +++ b/ckan/templates/home/index.html Thu Jul 28 14:10:23 2011 +0000 @@ -64,6 +64,7 @@ </py:if> <h4>Recently changed packages</h4> + <!-- ${c.latest_packages} --> ${package_list_from_dict(c.latest_packages)} <p><a href="${h.url_for(controller='revision', action='index', diff -r adf254b7c507 ckan/templates/layout_base.html --- a/ckan/templates/layout_base.html Sat Jul 23 18:16:15 2011 +0100 +++ b/ckan/templates/layout_base.html Thu Jul 28 14:10:23 2011 +0000 @@ -101,6 +101,7 @@ am_authorized_package_create = h.am_authorized(c, actions.PACKAGE_CREATE) ?> <li py:if="am_authorized_package_create">${h.nav_link(c, _('Add a package'), controller='package', action='new', id=None)}</li> + <li><a href="/storage/upload">Upload Data</a></li> <li>${h.nav_link(c, _('Tags'), controller='tag', action='index', id=None)}</li> <li>${h.nav_link(c, _('Groups'), controller='group', action='index', id=None, highlight_actions = 'new index')}</li> <li>${h.nav_link(c, _('About'), controller='home', action='about', id=None)}</li> diff -r adf254b7c507 ckan/templates/package/read.html --- a/ckan/templates/package/read.html Sat Jul 23 18:16:15 2011 +0100 +++ b/ckan/templates/package/read.html Thu Jul 28 14:10:23 2011 +0000 @@ -56,7 +56,7 @@ </py:if> <li class="widget-container widget_text"> - <py:if test="c.pkg.isopen() and c.pkg.resources"> + <py:if test="c.pkg.isopen and c.pkg.resources"> <h3> This Package is Open </h3> @@ -80,7 +80,7 @@ </p> </py:if> - <py:if test="not(c.pkg.isopen() and c.pkg.resources)"> + <py:if test="not(c.pkg.isopen and c.pkg.resources)"> <h3 i18n:msg="">This package is Not Open</h3> <p>Either because it is not openly licensed or is missing }}}
#1123 1311863798000000 nils.toedtmann Looks like the packages on apt.okfn.org are now architecture "all" - great. To avoid future confusion, i change this ticket's solution to "fixed".
#1245 1311864530000000 annapowellsmith Would be nice to list some extensions somewhere on this site (features page perhaps). Along the lines of: http://docs.ckan.org/extensions.html#finding-extensions It wasn't until I wrote this list that I realised just how many extensions there were, and what it said about the CKAN community that there were so many... So I think it would be a good sales point.
#1246 1311885360000000 pudo I'm afraid to say I don't understand any of this: the faulty display of openness is triggered as removing the brackets turns the isopen() check into a check for the existence of the method. Reverting the changes on the server seems to work without generating exceptions. What exactly is the issue that was supposed to be fixed originally?
#1247 1311955176000000 rgrp Done. See http://ckan.readthedocs.org/ and http://docs.ckan.org/ will propagate soon.
#1249 1311959356000000 pudo fixed in cset:0398243111d4
#1192 1312191462000000 rgrp Marking this as closed as Anna has completed and this is now deployed at http://docs.ckan.org/.
#1229 1312191530000000 amercader Done, except for Revision related stuff, which probably overlaps with work being done by David Raznick.
#1208 1312191646000000 pudo This should be closed: * https://github.com/okfn/webstore - See README for details * Python client lib: https://github.com/okfn/webstore-client * http://webstore.thedatahub.org/ - Demo install * http://wiki.ckan.net/Webstore - Overview
#816 1312192499000000 rgrp What about idea of 'model' refactor for resource, specifically to have: * format: human created format string with possible nesting e.g. zip:csv * kind: file | api | example | service * mimetype: standard mimetype (e.g. for zipped csv would be application/zip) * mimetype-inner: mimetype of innermost object (so for example would be text/csv) See http://lists.okfn.org/pipermail/ckan-discuss/2011-April/001139.html
#816 1312212782000000 dread I like rgrp's idea to change the model. But since it's complicated it would be good to have a UI. e.g. You might make format field simply a drop-down with only some common options: 'CSV, Excel, Zip, Other'. If you go for Other, then a more substantial UI appears, to cover more filetypes, APIs, services and a text box for Other. And if you select 'Zip' then another set of options appears to say what is in the zip. etc.etc. Another approach would be to have a background process downloading the data file and filling this in for you.
#816 1312295371000000 johnglover I have implemented the resource autocomplete so am closing this for now. I agree that more information on the resource would be a good addition. It seems a bit of a different issue to just adding autocomplete though, with potentially far more changes to the codebase, so maybe it would be better discussed in a separate ticket/CREP. However, any new fields would presumably want to either be constrained or have an autocomplete and so can reuse the work done on this feature. Cheers, John
#1250 1312370201000000 pudo The issue is caused after search returns from solr and tries to get objects from the database. These are by default ordered by name and we need to restore the original order after fetching them from the database. Fixed in module.
#1041 1312372367000000 rgrp James says this is not priority and he does not plan to do docs for debs until he has got that system functioning.
#1256 1312409296000000 kindly This has also caused problems in authorization for deleted packages.
#1239 1312453156000000 dread Fixed in 1.4.2 release. Broken in UI in 1.4.1 and API in 1.4 & 1.4.1.
#1140 1312491320000000 kindly fixed cset:987da68ea4f6 The package group table needed to trigger a reindex of the package.
#1140 1312537257000000 dread Broken since CKAN v1.1. I have added this fix to v1.4.2 and v1.4.3 releases.
#1265 1312566052000000 dread Fixed in cset:a1b8dd169f4e on 1.4.3 release and merged to default.
#1266 1312797477000000 dread Fixed in 987dc4fe078e on 1.4.3 branch and default.
#1256 1312813529000000 kindly fixed cset:357cf9377b25
#1258 1312813614000000 kindly cset:e22d4e385fc8
#1267 1312889130000000 dread John thinks it is a problem with WebOb 1.0.6
#1267 1312890962000000 dread This error has been occurring on at.ckan.net which has webob 0.9.7.1-1
#1267 1312894071000000 dread Running tests with WebOb 0.9.7.1, we see the error! Am now trying to fix.
#1267 1312897724000000 dread Fixed in cset:db5bf5b45b4f for ckan 1.4.3. Later versions of webob add the charset header for unicode errors (webob/exc.py:generate_response): {{{ extra_kw = {} if isinstance(body, unicode): extra_kw.update(charset='utf-8') resp = Response(body, status=self.status, headerlist=headerlist, content_type=content_type, **extra_kw ) }}} This was fixed in webob cset:25fc14d49623 which went into WebOb 1.0. This problem affects versions from CKAN 1.4.2 back to 1.3.1. We had been avoiding this in CKAN previously by only doing abort() with a plain string.
#1270 1312912188000000 dread Fixed in cset:ca5295c9c8cf and released as ckanclient version 0.9
#1172 1312991332000000 dread I've started this on branch defect-1172-exceptions
#1272 1313745867000000 rgrp Assume these are the 3 spatial fields listed on http://wiki.ckan.net/Geospatial_Capabilities. I'm a big +1 here. Starting point would be a bit more documentation on that wiki page. Questions: * Is this work going in ckanext-spatial? * What is being proposed here, I assume, is to automatically index those fields in our geospatial search. * Should we develop a simple display widget using google maps or similar (this is orthogonal to the WMS preview i think) which *just* uses the core fields. This is probably a separate ticket and possibly separate from ckanext-spatial extension.
#1271 1313753663000000 rgrp Closed in cset:749cb3a087e3
#1142 1313775665000000 rgrp This was completed 4 weeks ago and can now be marked as complete. See http://docs.ckan.org/ and http://wiki.ckan.net/ and http://ckan.org/
#1281 1314021919000000 dread Done by pudo in cset:eaf342823caf on default branch, so headed for release 1.4.4. "Work around the fact that locale changes do not affect the running request and therefore led to an incorrect flashing message. Now manually changing language."
#1262 1314029150000000 dread Pudo - I'm not sure what's meant here. Are you talking about moderation for new users? Or some other sort of enforcement? Is it spam you want to target? The links you're concerned about - are they the spam ones in the User's 'about' field? If so, I think we *should* allow users to have links, although continue with applying them with 'nofollow' and attack spammers back in a concerted strategy across all editable fields - see ticket #1257.
#1263 1314029279000000 dread This was fixed in cset:8d293393a9f2 for release 1.4.3.
#1226 1314029434000000 dread This appears to have gone away. No sign of it for two weeks on any OKF sites.
#1214 1314029628000000 dread This appears to have been fixed by version 1.4.3.
#1202 1314030352000000 dread The CKAN package page links to datapkg at: http://blog.okfn.org/2010/02/23/introducing-datapkg/ (it has always been this link). I think Rolf is referring to the three links to knowledgeforge.net in the blog post itself. These need to be edited by Rufus. Note datapkg has moved again to github.
#1175 1314030683000000 dread Hi Flavio, is this still a concern? I'd like to close this ticket unless you can provide details of the error from your logs or exception email.
#951 1314031006000000 dread Specific problem explained by having two users with the same name - one was authorized and one wasn't. See: http://lists.okfn.org/pipermail/ckan-dev/2011-April/000550.html
#947 1314031310000000 dread This was done in #1025 through configuration. Happily you can setup config in an extension, so covers this ticket's aims too.
#803 1314031451000000 dread We now use nose to test migrations instead.
#503 1314031851000000 dread Relationships exist now on ckan.net. e.g. http://ckan.net/package/rkb-explorer-ibm No current DGU interest.
#1275 1314110016000000 johnglover Completed in branch feature-1275-solr-search. Up to date with current default but not yet merged into default. John
#948 1314112642000000 dread Note: Packages will now be excluded from the search view (see #1283). Also there is now no redirect requirement from DGU any more.
#1172 1314118281000000 dread Is in cset:47225f03e2af and merged to default for 1.4.4. It now barfs instead of failing silently: * if the ckan config file has any errors * search indexing error There are some other "except Exception" statements
#1172 1314118883000000 dread I took out "raise" by mistake in ckan/lib/search/worker.py:26 but John has added it in in the corresponding place in ckan/lib/search/__init__.py on branch feature-1275-solr-search anyway.
#1154 1314120497000000 johnglover Fixed in core (branch feature-1275-solr-search).
#1154 1314177477000000 dread Looking at the changeset, it now does a connection check at start-up of CKAN. If it fails then CKAN runs but doesn't show any packages, apart from via the Tags interface, and the search page and widget on the front-page are removed. (BTW Is this a useful thing to do? Without search, perhaps CKAN should simply not start?) I believe the problem pudo is referring to is temporary periods when the SOLR server is slow/overloaded/down. During this time pudo and I get exception emails saying connection failed for request. e.g. {{{ Module ckan.controllers.home:29 in index << query = query_for(model.Package) query.run(query='*:*', facet_by=g.facets, limit=0, offset=0, username=c.user) c.facets = query.facets c.fields = [] >> limit=0, offset=0, username=c.user) Module ckan.lib.search.common:115 in run << self.query = QueryParser(query, terms, fields) self.query.validate() self._run() self._format_results() return {'results': self.results, 'count': self.count} >> self._run() Module ckanext.solr.backend:63 in _run << # this wrapping will be caught further up in the WUI. log.exception(e) raise SearchError(e) finally: conn.close() >> raise SearchError(e) SearchError: [Errno 111] Connection refused }}} (recently from nl.ckan.net)
#1154 1314178325000000 johnglover > Looking at the changeset, it now does a connection check at start-up of CKAN. If it fails then CKAN runs but doesn't show any > packages, apart from via the Tags interface, and the search page and widget on the front-page are removed. (BTW Is this a > useful thing to do? Without search, perhaps CKAN should simply not start?) When discussing the ticket we agreed that we should allow ckan to start without search. It is possible that people simply want to list/view information without searching for it and we decided not to prevent them from doing so. > I believe the problem pudo is referring to is temporary periods when the SOLR server is slow/overloaded/down. During this > time pudo and I get exception emails saying connection failed for request. Ok, other options then: * if an error occurs during search, return 0 results * if an error occurs during search, redirect to a generic 'search is unavailable' page * if an error occurs during search, retry the query a certain number of times before giving up and doing one of the above. What do you suggest?
#1154 1314179290000000 dread Search requests - option one seems like a bad user experience, and option three seems too complicated. I should do option 2, explain it may well be a temporary problem, and let the user press reload, as per any normal web request. A more difficult problem though is what to do about a failed search indexing. Because of the try/except catch we've had until the patch yesterday, we get no exception emails for this, but I'm worried that this has caused packages to not be indexed, and hence are essentially lost in CKAN (unless someone realises there is a problem and does a reindex at some point). BTW I added paster command 'search-index check' for looking at this problem, but I don't think it was ever added to solr. Is it easy to add get_all_entity_ids for the purpose of checking for this problem?
#1240 1314180488000000 dread I basically agree, but just thought I'd add a couple of comments into the mix. > GET at root returns whole package set (a *bad* idea) Sure it has a cost, but I don't see what the problem with this is if the list is cached. Forcing developers to paging through the list of packages is putting unnecessary obligations. I'd say that the top two RESTful operations are listing objects and getting a particular one, and you'd be taking away one of those. > Unify on /api/v{version num}/... structure Where is it not currently unified?
#1154 1314191496000000 johnglover Ok I'll implement option 2 for search query issues. Yes it is easy to get a list of IDs from Solr so I'll have a look at that command and make sure that it works. I think a case could be made for not allowing a package to be added if it cannot be indexed however, and making the user simply retry the request.
#1154 1314192324000000 dread Yes, forcing a retry may be possible, although I guess on these occasions you'd be looking at rolling back the creation of the package object too, which is not really in the spirit of the notifications system at present. This is why we had the queue for solr for occasions like this - is it here no longer?
#1154 1314192753000000 johnglover Not in core, there is a queue feature in the extension still which I could add to core. I think though it was a case of using either the queue or synchronous search, I don't think it would automatically fall back to the queue if there was an indexing problem, but we could look at that. Was there not talk of changing the queue system recently though? There are also no tests for the Solr queue so that would take a bit of work as well.
#1154 1314193633000000 johnglover RE search queries failing: Just having a look through the controller code and the package search controller and the search api both perform option 2 already, so I'll just update the home controller.
#1154 1314195684000000 dread Ok, I chatted to David about this and we feel it should go like this: notification to search indexer fails and raises an exception, this gets passed through the notification system back up to the code that creates the package, and you do a roll back. Does this tally with what you mentioned in the last comment or is what's there better? Thanks for the info on the queue and ckanext-solr. It sounds like this isn't deployed successfully anywhere (or does pudo know if it is?). In this case we should be clear that it is not what it says it is: "This extension adds support for package search using Apache Solr to CKAN" and maybe simply mark it as deprecated. Pudo what do you think?
#1154 1314196683000000 johnglover > Ok, I chatted to David about this and we feel it should go like this: notification to search indexer fails and raises an exception, > this gets passed through the notification system back up to the code that creates the package, and you do a roll back. Does this > tally with what you mentioned in the last comment or is what's there better? This sounds good to me, definitely better than what is currently there. > Thanks for the info on the queue and ckanext-solr. It sounds like this isn't deployed successfully anywhere (or does pudo > know if it is?). I'm not actually sure if the queue works or not, but if it does I'll have to write some tests for it before we could add it to core. A working queue system would be useful generally however, as there are other things that it would be useful for (such as QA), but yes, going forward the Solr extension should really be deprecated.
← 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 →
Note: See TracReports for help on using and creating reports.