{22} Trac tickets (2647 matches)

Results (1201 - 1300 of 2647)

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Id Type Owner Reporter Milestone Status Resolution Summary Description Posixtime Modifiedtime
#1210 defect dread dread ckan-v1.5-sprint-4 closed fixed POST application/json error handling with newer WebOb

WebOb? from v1.0.7 has some interesting new behaviour with reading request data for different Content-Types:

  • It seems that looking at request.body mangles request.POST. See fix in ckan cset:e83bad9caa1b (problem with WebOb? v1.0.7 but may go back to v1.0.4?)
  • A particular request caused an AssertionError? during looking at request.body, so we need to enclose this in a try/except. I couldn't recreate it in a test unfortunately (problem with WebOb? v1.0.7 but may go back to v1.0.2?)

Example:

Module ckan.controllers.api:206 in create
<<          log.debug('create: %s' % (context))
               try:
                   request_data = self._get_request_data()
               except ValueError, inst:
                   response.status_int = 400
>>  request_data = self._get_request_data()
Module ckan.lib.base:149 in _get_request_data
<<          cls.log.debug('Retrieving request params: %r' % request.params)
               cls.log.debug('Retrieving request POST: %r' % request.POST)
               cls.log.debug('Retrieving request POST body: %r' % request.body)
               if request.POST:
                   try:
>>  cls.log.debug('Retrieving request POST body: %r' % request.body)
Module paste.registry:137 in __getattr__
<<      
           def __getattr__(self, attr):
               return getattr(self._current_obj(), attr)
           
           def __setattr__(self, attr, value):
>>  return getattr(self._current_obj(), attr)
Module webob.request:470 in _body__get
<<          Return the content of the request body.
               """
               self.make_body_seekable() # we need this to have content_length
               r = self.body_file.read(self.content_length)
               self.body_file.seek(0)
>>  self.make_body_seekable() # we need this to have content_length
Module webob.request:697 in make_body_seekable
<<              self.body_file_raw.seek(0)
               else:
                   self.copy_body()
>>  self.copy_body()
Module webob.request:714 in copy_body
<<                  self.body = self.body_file_raw.read(length)
               elif self.is_body_readable:
                   self.body = self.body_file_raw.read()
                   self._copy_body_tempfile()
               else:
>>  self.body = self.body_file_raw.read()
Module webob.request:1190 in read
<<      def read(self, size=-1):
               body = self._get_body()
               if size < 0:
                   v = body[self.position:]
>>  body = self._get_body()
Module webob.request:1207 in _get_body
<<                  self._body = _encode_multipart(self.vars, self.content_type)
                   else:
                       assert 0, ('Bad content type: %r' % self.content_type)
               return self._body
>>  assert 0, ('Bad content type: %r' % self.content_type)
AssertionError: Bad content type: '; charset=utf-8'
1309949451000000 1309974781000000
#1211 enhancement kindly kindly ckan-v1.5 closed fixed [super] Drupal integration

Sub-tasks / tickets:

  • #1335 Action API - access to any logic layer function
  • Standardize logic functions on ordering: (context, data_dict) and put parameters into data_dict.
  • Add package_create_validate & package_update_validate logic functions
  • Allow extensions to provide logic layer functions (IActions)
1309958442000000 1315948703000000
#1214 defect dread dread closed worksforme API improvements (following javascript use)

Notes on the CKAN API from Aron:

  • Server returns text/html for errors even when "Accept: application/json" header is set.
  • DELETE package request requires a Content-Length? Should be 405 Method Not Allowed?
  • Tag returned as a JSON object when updating but as a string when requesting. Same with "extras" content.
  • How to remove key from "extras" object? Passing None as described in the docs is invalid JSON and using null doesn't work.
  • Tags return package names rather than ids when querying GET /tag/{id}
  • Doesn't return rendered_text property on package update.
  • Relationships use "object" key rather than id or package_id.
1309975222000000 1314029628000000
#1215 defect dread closed fixed Edit Resources table - removing all rows prevents adding them again

I'm not sure if this was reported earlier, but I have found a minor bug in the "Add new package" form. As far as I can see this bug exists in v1.3.2 and also in v1.4.1a (that runs on ckan.net). This is a problem with adding new resources in "Resources" section. To reproduce the bug do the following:

  1. Go to "Add new package", for instance http://ckan.net/package/new
  2. Scroll down to "Resources" section.
  3. Click on "Remove this row" link (red circle).
  4. Say "Yes" to "Are you sure you wish to remove this row?"
  5. Now there should be no rows. Only the header "URL* Format Description"
  6. Click on "Add row to table"
  7. All you get now is another row with table header "URL* Format Description". You will not be able to add any resources now. Solution is to reload the page.

The simplest solution to this bug will be to hide the "Remove this row" link if there is only one row.

1310133224000000 1310385485000000
#1217 defect dread dread ckan-sprint-2011-11-07 closed invalid Diff exception

Exception raised doing this particular diff.

http://ckan.net/revision/diff/sudocfr?diff_entity=package&oldid=a18b2520-09e0-44be-80dd-4fea94ffe7d1&diff=3934ff16-a3a7-4481-b390-72944903b945

...
Module ckan.controllers.revision:142 in diff
<<          if c.diff_entity == 'package':
                   c.pkg = model.Package.by_name(id)
                   diff = c.pkg.diff(c.revision_to, c.revision_from)
               elif c.diff_entity == 'group':
                   c.group = model.Group.by_name(id)
>>  diff = c.pkg.diff(c.revision_to, c.revision_from)
Module ckan.model.package:449 in diff
<<                              display_id = to_obj_rev.tag.name
                               elif obj_class.__name__ == 'PackageExtra':
                                   display_id = to_obj_rev.key
                               else:
                                   display_id = related_obj_id[:4]
>>  display_id = to_obj_rev.key
AttributeError: 'NoneType' object has no attribute 'key'

Seems to be to do with this particular extra key:

[<PackageExtraRevision id=ba1f3b3e-f61f-4924-bae0-ec3dd2163ae4 package_id=febe4da9-5874-4411-a7da-b497fdebf739 key=namespace value=http://www.sudoc.fr/ state=active revision_id=5ec8e730-c73d-4f1c-989b-2e015eb3d315 continuity_id=ba1f3b3e-f61f-4924-bae0-ec3dd2163ae4 expired_id=None revision_timestamp=2011-07-08 13:48:11.027260 expired_timestamp=9999-12-31 00:00:00 current=True>]

Is it because it was created between the from_revision and to_revision?

1310375498000000 1320663277000000
#1218 enhancement dread [email protected] ckan-sprint-2011-10-28 closed fixed Colour the History tab icon

Change the color of the "History" tab icon to yellowish, like the rest of the icons in the other tabs ("View" and "Edit", "Authorization").

Reasoning

When you view a data package, for instance http://ckan.net/package/thesaurus-w, you will see "View", "Edit", "History" tabs on the top. "History" tab has a black-and-white icon which makes it look like an inactive/disabled tab (since the text is also grayed out when the tab is not selected). This is not a major issue, but it is a little bit confusing for the users. This icon exists in v1.3.2 and also in v1.4.1a (that runs on ckan.net).

The simple solution would be to change the color of the "History" tab icon and give it the same yellowish color like the rest of the icons in the other tabs (e.g. "View" and "Edit").

1310375768000000 1310389390000000
#1219 defect timmcnamara closed fixed CSS issues on IE7

As reported on ckan-dev:

items in the footer of CKAN ("Packages", "Groups & Tags", "About", "Language", etc.) are shown vertically instead of horizontally in IE7. This works fine in later browsers like IE8, IE9, FF4, and latest Opera and Chrome.

This seems to exist in all recent CKAN versions up to 1.4.1a.

1310423688000000 1310740534000000
#1221 defect dread dread ckan-sprint-2011-10-28 closed fixed Password reset tidy up
  • No tests.
  • 500 error given when user is not found.
1310475661000000 1310556607000000
#1222 defect dread dread ckan-sprint-2011-10-28 closed fixed Mailer has no tests 1310479136000000 1310556618000000
#1223 enhancement pudo pudo closed fixed Caching of static files

StaticURLParser can have caching - use it

1310573854000000 1310573893000000
#1225 enhancement dread dread ckan-sprint-2011-10-28 closed fixed Tool for mass tag changes

Using ckanapi

1310642161000000 1310642407000000
#1226 defect dread closed worksforme Genshi stream exception

IndexError? exception in genshi.filters.i18n keeps popping up daily.

e.g.

...
Module ckan.controllers.authorization_group:28 in index
<<              items_per_page=20
               )
               return render('authorization_group/index.html')
       
           def read(self, id):
>>  return render('authorization_group/index.html')
Module ckan.lib.base:74 in render
<<      
           return cached_template(template_name, render_template, cache_key=cache_key, 
                                  cache_type=cache_type, cache_expire=cache_expire)
                                  #, ns_options=('method'), method=method)
>>  cache_type=cache_type, cache_expire=cache_expire)
Module pylons.templating:249 in cached_template
<<          return content
           else:
               return render_func()
>>  return render_func()
Module ckan.lib.base:63 in render_template
<<              stream = item.filter(stream)
               
               return literal(stream.render(method=method, encoding=None))
           
           if 'Pragma' in response.headers:
>>  return literal(stream.render(method=method, encoding=None))
Module genshi.core:183 in render
<<              method = self.serializer or 'xml'
               generator = self.serialize(method=method, **kwargs)
               return encode(generator, method=method, encoding=encoding, out=out)
       
           def select(self, path, namespaces=None, variables=None):
>>  return encode(generator, method=method, encoding=encoding, out=out)
Module genshi.output:57 in encode
<<          _encode = lambda string: string
           if out is None:
               return _encode(''.join(list(iterator)))
           for chunk in iterator:
               out.write(_encode(chunk))
>>  return _encode(''.join(list(iterator)))
Module genshi.output:339 in __call__
<<          for filter_ in self.filters:
                   stream = filter_(stream)
               for kind, data, pos in stream:
                   cached = cache_get((kind, data))
                   if cached is not None:
>>  for kind, data, pos in stream:
Module genshi.output:670 in __call__
<<          _gen_prefix = _gen_prefix().next
       
               for kind, data, pos in stream:
                   output = cache_get((kind, data))
                   if output is not None:
>>  for kind, data, pos in stream:
Module genshi.output:771 in __call__
<<          push_text = textbuf.append
               pop_text = textbuf.pop
               for kind, data, pos in chain(stream, [(None, None, None)]):
       
                   if kind is TEXT:
>>  for kind, data, pos in chain(stream, [(None, None, None)]):
Module genshi.output:586 in __call__
<<      def __call__(self, stream):
               prev = (None, None, None)
               for ev in stream:
                   if prev[0] is START:
                       if ev[0] is END:
>>  for ev in stream:
Module genshi.core:288 in _ensure
<<      # unchanged
           yield event
           for event in stream:
               yield event
>>  for event in stream:
Module genshi.core:288 in _ensure
<<      # unchanged
           yield event
           for event in stream:
               yield event
>>  for event in stream:
Module genshi.filters.transform:686 in _unmark
<<      def _unmark(self, stream):
               for mark, event in stream:
                   kind = event[0]
                   if not (kind is None or kind is ATTR or kind is BREAK):
>>  for mark, event in stream:
Module genshi.filters.transform:1145 in __call__
<<          :param stream: The marked event stream to filter
               """
               for mark, event in stream:
                   yield mark, event
                   if mark is ENTER:
>>  for mark, event in stream:
Module genshi.filters.transform:714 in __call__
<<          stream = iter(stream)
               next = stream.next
               for mark, event in stream:
                   if mark is None:
                       yield mark, event
>>  for mark, event in stream:
Module genshi.filters.transform:682 in _mark
<<      def _mark(self, stream):
               for event in stream:
                   yield OUTSIDE, event
>>  for event in stream:
Module genshi.core:288 in _ensure
<<      # unchanged
           yield event
           for event in stream:
               yield event
>>  for event in stream:
Module genshi.filters.transform:686 in _unmark
<<      def _unmark(self, stream):
               for mark, event in stream:
                   kind = event[0]
                   if not (kind is None or kind is ATTR or kind is BREAK):
>>  for mark, event in stream:
Module ckanext.googleanalytics.plugin:62 in download_adder
<<              (downloaded %s times)</span>'''
                   count = None
                   for mark, (kind, data, pos) in stream:
                       if mark and kind == START:
                           href = data[1].get('href')
>>  for mark, (kind, data, pos) in stream:
Module genshi.filters.transform:714 in __call__
<<          stream = iter(stream)
               next = stream.next
               for mark, event in stream:
                   if mark is None:
                       yield mark, event
>>  for mark, event in stream:
Module genshi.filters.transform:682 in _mark
<<      def _mark(self, stream):
               for event in stream:
                   yield OUTSIDE, event
>>  for event in stream:
Module genshi.core:288 in _ensure
<<      # unchanged
           yield event
           for event in stream:
               yield event
>>  for event in stream:
Module genshi.filters.transform:686 in _unmark
<<      def _unmark(self, stream):
               for mark, event in stream:
                   kind = event[0]
                   if not (kind is None or kind is ATTR or kind is BREAK):
>>  for mark, event in stream:
Module genshi.filters.transform:1175 in __call__
<<          """
               callable_value = hasattr(self.value, '__call__')
               for mark, (kind, data, pos) in stream:
                   if mark is ENTER:
                       if callable_value:
>>  for mark, (kind, data, pos) in stream:
Module genshi.filters.transform:714 in __call__
<<          stream = iter(stream)
               next = stream.next
               for mark, event in stream:
                   if mark is None:
                       yield mark, event
>>  for mark, event in stream:
Module genshi.filters.transform:682 in _mark
<<      def _mark(self, stream):
               for event in stream:
                   yield OUTSIDE, event
>>  for event in stream:
Module genshi.core:288 in _ensure
<<      # unchanged
           yield event
           for event in stream:
               yield event
>>  for event in stream:
Module genshi.filters.transform:686 in _unmark
<<      def _unmark(self, stream):
               for mark, event in stream:
                   kind = event[0]
                   if not (kind is None or kind is ATTR or kind is BREAK):
>>  for mark, event in stream:
Module genshi.filters.transform:1145 in __call__
<<          :param stream: The marked event stream to filter
               """
               for mark, event in stream:
                   yield mark, event
                   if mark is ENTER:
>>  for mark, event in stream:
Module genshi.filters.transform:714 in __call__
<<          stream = iter(stream)
               next = stream.next
               for mark, event in stream:
                   if mark is None:
                       yield mark, event
>>  for mark, event in stream:
Module genshi.filters.transform:682 in _mark
<<      def _mark(self, stream):
               for event in stream:
                   yield OUTSIDE, event
>>  for event in stream:
Module genshi.template.base:618 in _include
<<                      tmpl = self.loader.load(href, relative_to=event[2][0],
                                                   cls=cls or self.__class__)
                           for event in tmpl.generate(ctxt, **vars):
                               yield event
                       except TemplateNotFound:
>>  for event in tmpl.generate(ctxt, **vars):
Module genshi.template.base:618 in _include
<<                      tmpl = self.loader.load(href, relative_to=event[2][0],
                                                   cls=cls or self.__class__)
                           for event in tmpl.generate(ctxt, **vars):
                               yield event
                       except TemplateNotFound:
>>  for event in tmpl.generate(ctxt, **vars):
Module genshi.template.base:618 in _include
<<                      tmpl = self.loader.load(href, relative_to=event[2][0],
                                                   cls=cls or self.__class__)
                           for event in tmpl.generate(ctxt, **vars):
                               yield event
                       except TemplateNotFound:
>>  for event in tmpl.generate(ctxt, **vars):
Module genshi.template.base:605 in _include
<<          from genshi.template.loader import TemplateNotFound
       
               for event in stream:
                   if event[0] is INCLUDE:
                       href, cls, fallback = event[1]
>>  for event in stream:
Module genshi.template.markup:378 in _match
<<                      for event in self._match(self._flatten(template, ctxt,
                                                                  **vars),
                                                    ctxt, start=idx + 1, **vars):
                               yield event
>>  ctxt, start=idx + 1, **vars):
Module genshi.template.markup:327 in _match
<<                      break
       
               for event in stream:
       
                   # We (currently) only care about start and end events for matching
>>  for event in stream:
Module genshi.template.base:545 in _flatten
<<          while 1:
                   for kind, data, pos in stream:
       
                       if kind is START and data[1]:
>>  for kind, data, pos in stream:
Module genshi.filters.i18n:177 in _generate
<<                  msgbuf.append(*previous)
                       previous = None
                   for event in msgbuf.translate(gettext(msgbuf.format())):
                       yield event
                   if previous:
>>  for event in msgbuf.translate(gettext(msgbuf.format())):
Module genshi.filters.i18n:1029 in translate
<<                      )
       
               parts = parse_msg(string)
               parts_counter = {}
               for order, string in parts:
>>  parts = parse_msg(string)
Module genshi.filters.i18n:1143 in parse_msg
<<      if string:
               parts.append((stack[-1], string))
       
           return parts
>>  parts.append((stack[-1], string))
IndexError: list index out of range
1311008861000000 1314029434000000
#1227 enhancement timmcnamara ckan-backlog new Display packages' tags in search results

In when displaying search results, it would be useful to also display the tags of a package. Sometimes it's difficult to infer the scope of what the package does from the title and the first sentence of the description. Tags are quite concise way to display rich information.

ENV=datacatalos.org, with CKAN 1.4.2a

1311034262000000 1339774147000000
#1228 enhancement dread dread ckan-sprint-2011-10-28 closed fixed Display OpenID login errors

When there are errors when logging into OpenID, these don't get displayed. You just get the login screen again.

1311086562000000 1311325161000000
#1229 enhancement dread thejimmyg ckan-sprint-2011-10-24 closed fixed Refactor all database code out of the controllers and into the logic layer

Progress can be tracked in this document:

https://spreadsheets.google.com/spreadsheet/ccc?key=0Atp3cZFjuIOAdFQzRnVoUHV6ckoyblI5TUdEcnJLVlE&hl=en_US#gid=0

1311093924000000 1319645778000000
#1230 enhancement kindly ckan-sprint-2011-10-28 closed fixed Make sure plugins tables work in tests.

Dgu tests failed as harvesting table are not created at the correct point.

1311100811000000 1311154142000000
#1231 requirement kindly thejimmyg ckan-backlog closed wontfix [super] Management Information Reporting

Child tickets:

  • #1101 Integrate stats and googlanalytics into site nav

We have a spreadsheet from UKLP of statistics we'd like to generate

1311173919000000 1325474447000000
#1232 enhancement thejimmyg ckan-backlog new [super] Interface improvements

Child tickets:

  • #1194 "Welcome back" message for newly registered user
  • #1202 Links to datapkg utility don't lead to info about it
  • #925 Change the search box icon to remove the down arrow
  • #923 Search box doesn't work in leaderboard page in stats extension
  • #1034 Flash message cached
  • #737 Markdown syntax summary page
  • #811 Extra field editing form layout breaks when there are long field names
1311178296000000 1315948536000000
#1233 enhancement thejimmyg ckan-backlog new [super] Improve wiki-style functionality for history

At the moment we have a good revisioning system but a poor history interface. We need to improve this in a number of areas:

  • #191 Searching by modification date
  • #193 Searching by time-related field
  • #301 Package discussion pages
  • #1236 Package history page should provide links to pages at particular revisions, similar to the wikipedia pages
  • #1236 Viewing old revisions or unmoderated changes should have a message at the top of the package page
  • Other improvements as per my word doc.
1311179392000000 1315948668000000
#1234 enhancement dread dread ckan-sprint-2011-10-28 closed fixed Setting locale error conditions handled badly

Needs tests for error conditions for bad locale and redirect parameters.

1311179871000000 1311183328000000
#1235 enhancement thejimmyg ckan-future new [super] Search Improvements

Child tickets:

  • #234 UI Review - Autocomplete package names & tags in search
  • #193 Searching by time-related field
  • #191 Searching by modification date
  • #905 Unable to search with accented characters in package names
  • #906 Ability to search without accents for accented words
  • #924 Search box has no search button

Broadly speaking though we need to choose PostgreSQL, Solr or something else. We don't want to invest our time maintaining two search backends with a limited abstraction layer between the two.

1311182641000000 1311182641000000
#1236 enhancement dread dread ckan-sprint-2011-11-07 closed fixed View a package at a given revision

As a user I want to view a package at a given revision:

  • When I visit /package/[email protected] or /package/[email protected] I should be shown package at date yyyy-mm-dd or revision rrrrrrr.
  • viewing this page, it should say it is the old revision, just like wikipedia
  • package history page should provide links to these pages
1311269333000000 1330019788000000
#1238 defect kindly dread ckan-sprint-2011-10-28 closed invalid Viewing a package @date misses non-PackageRevisions

There's a problem with getting old versions of packages using @date. The query looks for the PackageRevision? related to the date, but ignores TagRevisions? etc.

So for example if you compare this table: http://ckan.net/package/history/osm with this view: http://ckan.net/package/osm%402010-11-30%2000%3A21%3A49.627830 you actually see the 2010-01-13 11:13 revision - the wrong revision.

1311351303000000 1311759784000000
#1239 defect rgrp rgrp ckan-sprint-2011-10-28 closed fixed Remove and re-add tag on package not working

See reports on list at:

Tracked this to issue in dictization/model_save whereby was ignoring case where Package Tag already there but in deleted state. Rather than describe at length see fix.

1311441020000000 1312453156000000
#1240 enhancement kindly rgrp ckan-backlog assigned [super] API v4

(Just creating this ticket as somewhere to keep notes)

  • Decide on REST api versus action API
    • Do we want to support both?
  • Tidying
    • Unify on /api/v{version num}/... structure (do we want a default option that points to current default? e.g. /api/default/ ...)
    • extras merged into normal field list in package
    • Get rid of /rest/ so just have api/v1/package
    • Get rid of separation of search api from 'rest' api
      • Propose that GET on REST index is search e.g. /package/?q=...
        • This is also resolves issue whereby GET at root returns whole package set (a *bad* idea) as this would now become the matchall search query (with a default limit on items returned)
  • Resource read/write in API (separate from package)
    • Does this need authorization work?
  • user/account API - read/write
  • Remove autocomplete -- can just use search
    • Do not worry about backwards compat as should only be used in our js (if others using it too bad!)
1311525660000000 1325473312000000
#1241 task timmcnamara timmcnamara closed wontfix Overview video of http://thedatahub.org

Produce a video explaining some of the things which can be done on the datahub.org

1311567813000000 1340632688000000
#1242 enhancement timmcnamara closed fixed thedatahub.org's login page

The right hand side of thedatahub.org login screen is a little confusing:

    Not a member?

    Join CKAN to contribute packages under your own name.

     - Register
     - Reset your password
     - Privacy Policy

Suggested improvements to make:

  • replace CKAN with thedatahub.org
  • in general, edit the copy to be more inviting
  • remove "Reset your password" from that bubble
  • move the privacy policy link to somewhere less prominent
1311569218000000 1323173292000000
#1243 enhancement timmcnamara closed invalid [thedatahub.org] "Upload Data" should provide a nicer

If someone who is not logged in clicks on "Upload Data", they are presented with

    Not authorized to upload files.

This is in emboldened red text. That is, the system is telling a user that they're doing something illegitimate, even though they're actually following an invitation from the main menu.

I propose two options for resolving this:

  1. only display the "Upload Data" tab if someone is authorised to upload data
  2. if someone isn't logged int, provide a nice welcome message thanking them for wanting to upload data, but we only allow registered users to do so

ENV: thedatahub.org, CKAN 1.4.2b

1311577466000000 1328000405000000
#1244 enhancement dread assigned Notes field carriage-returns converted to CRLF

When you edit a package in the web form, if the notes field had \n as the End Of Line symbol, it gets lost when you preview or save the package, and the notes field is displayed all on one line.

This can be seen when editing annakarenina (as created by 'paster create-test-data'). The diff shows for example:

- Some test notes
+ Some test notes
?                +

but it would more clearly be shown as:

- Some test notes\n
+ Some test notes
?                ++

This is a significant problem with DGU, since a lot comes in via the API.

It's not clear what we should do about it. We could standardise on \n or \r\n when the form submission comes in. Do different browsers on different platforms do different things with EOLs?

Analysis

Displaying the package: the Markdown processor respects both EOLs when displaying the field, putting each line in a <p> tag.

Creating the package edit form: placed into <textfield>.

Browser displaying package edit form: <textfield> displays \n and \r\n as EOL. But \n\n gets compressed to one EOL. But on submission, both are returned as \r\n.

Receiving the edited package: Somewhere along the line the EOL gets converted to \n\n.

1311689456000000 1340191253000000
#1245 task rgrp ckan-v1.5 closed fixed Reorganise ckan.org pages

Here is Rufus' plan for ckan.org reorganisation from http://ckan.okfnpad.org/documentation (22/7/11) (ticket made by dread):

/solutions
  /overview <--- current 'about' page (the software)
  /customers <-- say "in addition to our clients..."
  /features <-- needs most work (pictures etc)
/pricing/  <-- simplest thing possible ... talk with Ira (cover pricing and support)
/developers <--- links to http://docs.ckan.org/, getting the software, the wiki, the mailing list /contact
/blog

Tasks:

  • add tagline in header - "the data portal software"
  • make and add glossy PDF
  • change "data hub" to "data portal" passim (if only using one term)
  • add [email protected]
  • image banner on home page - redo screenshots without menus
1311861708000000 1317422640000000
#1246 defect pudo [email protected] ckan-sprint-2011-10-28 closed fixed Search results on ckan.net are mistakenly all 'open'

All package search results on ckan.net are labelled as 'open' even when their license is closed or unknown: http://ckan.net/package

1311863353000000 1311892816000000
#1247 task rgrp dread ckan-sprint-2011-10-28 closed fixed Put docs on ReadTheDocs

Put docs on ReadTheDocs? instead of eu13.

1311941948000000 1311955185000000
#1248 defect dread dread ckan-sprint-2011-11-07 closed fixed License exception

Exception received on test.ckan.net at 10.12 on 29/7/11

http://test.ckan.net/package?__cache=2759373&page=4
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
           
           def _inspect_call(self, func):
>>  return func(**args)
Module ckan.controllers.package:153 in search
<<                          }
       
                   query = get.package_search(context,data_dict)
       
                   c.page = h.Page(
>>  query = get.package_search(context,data_dict)
Module ckan.logic.action.get:514 in package_search
<<      for package in query.results:
               result_dict = table_dictize(package, context)
               result_dict = _extend_package_dict(result_dict,context)
       
               results.append(result_dict)
>>  result_dict = _extend_package_dict(result_dict,context)
Module ckan.logic.action.get:537 in _extend_package_dict
<<      license_id = package_dict['license_id']
           if license_id:
               isopen = model.Package.get_license_register()[license_id].isopen()
               package_dict['isopen'] = isopen
           else:
>>  isopen = model.Package.get_license_register()[license_id].isopen()
Module ckan.model.license:59 in __getitem__
<<              return default
               else:
                   raise KeyError, "License not found: %s" % key
       
           def get(self, key, default=None):
>>  raise KeyError, "License not found: %s" % key
KeyError: u'License not found: None'

and a similar one:

 http://test.ckan.net/tag/barbecue
Module pylons.wsgiapp:324 in dispatch
<<          if log_debug:
                   log.debug("Calling controller class with WSGI interface")
               return controller(environ, start_response)
           
           def load_test_env(self, environ):
>>  return controller(environ, start_response)
Module ckan.lib.base:118 in __call__
<<          # available in environ['pylons.routes_dict']    
               try:
                   return WSGIController.__call__(self, environ, start_response)
               finally:
                   model.Session.remove()
>>  return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:221 in __call__
<<                  return response(environ, self.start_response)
               
               response = self._dispatch_call()
               if not start_response_called:
                   self.start_response = start_response
>>  response = self._dispatch_call()
Module pylons.controllers.core:172 in _dispatch_call
<<              req.environ['pylons.action_method'] = func
                   
                   response = self._inspect_call(func)
               else:
                   if log_debug:
>>  response = self._inspect_call(func)
Module pylons.controllers.core:107 in _inspect_call
<<                        func.__name__, args)
               try:
                   result = self._perform_call(func, args)
               except HTTPException, httpe:
                   if log_debug:
>>  result = self._perform_call(func, args)
Module pylons.controllers.core:60 in _perform_call
<<          """Hide the traceback for everything above this method"""
               __traceback_hide__ = 'before_and_this'
               return func(**args)
           
           def _inspect_call(self, func):
>>  return func(**args)
Module ckan.controllers.tag:2 in read
Module ckan.lib.cache:167 in wrapper
<<      log = __import__("logging").getLogger("proxy_cache")
           def wrapper(func, *args, **kwargs):
               result = func(*args, **kwargs)
       
               pylons = get_pylons(args)
>>  result = func(*args, **kwargs)
Module ckan.controllers.tag:64 in read
<<          data_dict = {'id':id}
               try:
                   c.tag = get.tag_show(context,data_dict)
               except NotFound:
                   abort(404, _('Tag not found'))
>>  c.tag = get.tag_show(context,data_dict)
Module ckan.logic.action.get:341 in tag_show
<<      extended_packages = []
           for package in tag_dict['packages']:
               extended_packages.append(_extend_package_dict(package,context))
       
           tag_dict['packages'] = extended_packages
>>  extended_packages.append(_extend_package_dict(package,context))
Module ckan.logic.action.get:537 in _extend_package_dict
<<      license_id = package_dict['license_id']
           if license_id:
               isopen = model.Package.get_license_register()[license_id].isopen()
               package_dict['isopen'] = isopen
           else:
>>  isopen = model.Package.get_license_register()[license_id].isopen()
Module ckan.model.license:59 in __getitem__
<<              return default
               else:
                   raise KeyError, "License not found: %s" % key
       
           def get(self, key, default=None):
>>  raise KeyError, "License not found: %s" % key
KeyError: u'License not found: CreativeCommonsAttributionCCBY25'
1311954136000000 1319797763000000
#1249 enhancement pudo rgrp ckan-v1.5 closed fixed Exclude script tag from extraction for i18n

Currently have this script section put in for i18n. It shouldn't be.

<script type="text/javascript">
//<![CDATA[
(function($){
    $.fn.ajaxCreateSlug = function(name, url) {
        var title = this;
        var updater = {
            init: function(title, name) {
                // Add a new element where the validity of the package name can be displayed
                this.name_field = name;
                this.title_field = title;
                this.name_field.parent().append('<div id="package_name_valid_msg"></div>');
                this.title_field.blur(this.title_change_handler())
                this.title_field.keyup(this.title_change_handler())
                this.name_field.keyup(this.name_change_handler());
                this.name_field.blur(this.name_blur_handler());
                this.url = url;
            },
....
1311958536000000 1311959356000000
#1250 enhancement pudo rgrp ckan-v1.5 closed fixed Search results should be sorted by score rather than alphabetical

At the moment we sort search results alphabetically. While this is useful for doing 'browse' case where no search bad for all other cases.

Adopt default sort order of 'score' though may wish to keep alphabetical for no search term (i.e. wildcard).

Options:

  • Default this in solr (no need to touch code) but fragile and affects everything ...
  • Do it in code and default to score
  • Do it in code and have alphabetical (on name or title?) when no criteria otherwise score

Aside: may also wish to support search in query api but that is for later!

1311964752000000 1312370201000000
#1251 enhancement rgrp rgrp ckan-backlog closed fixed Rename wiki.ckan.net to wiki.ckan.org

Inline with general naming policy going forward.

1311965147000000 1317315159000000
#1252 enhancement rgrp rgrp ckan-v1.5 closed fixed Deprecate ckan.net in favour of thedatahub.org

Deprecate ckan.net in favour of thedatahub.org

  • Announce
  • Create thedatahub.org and have it run same data as ckan.net
  • Announce closing down of ckan.net with 302
  • Implement 302 permanent

Also redirect wiki.ckan.net to wiki.ckan.org

1311965339000000 1318160334000000
#1253 enhancement amercader amercader ckan-sprint-2011-10-28 closed fixed Authz refactor

We agreed on a system that allows overriding the authz checks at a logic function level (e.g. package_create, user_show) and supports the old (current) authz system.

1312199491000000 1313401367000000
#1254 task dread dread ckan-sprint-2011-10-28 closed fixed Feature sheet

Compose a feature list for CKAN. Use the existing Feature Matrix. Basic explanation of terms. For use by customers.

1312211925000000 1313401629000000
#1255 enhancement kindly kindly ckan-backlog assigned Drupal consistancy checks.

Make a robust way to make sure the drupal database is consistent with the ckan data.

1312219968000000 1313400054000000
#1256 defect kindly kindly ckan-sprint-2011-10-28 closed fixed fix modification extension for moderation

Solr is not working for moderation, pending changes are not being indexed.

1312280026000000 1319812556000000
#1257 enhancement dread ckan-backlog new Anti-Spam tools

We are getting more and more spam on ckan.net and we need to improve our strategy of combating it. It is bad because google ranks who we link to (which we do want for legitimate links), and our front page contains the latest package edits, so spam is immediately very visible.

Spam:

  • Package creation
  • Package edit
  • User creation
  • Group creation

Systems to consider:

  • Automatic
    • captcha
    • bayesian scoring
  • Sysadmin
    • a tool to quickly analyse and remove packages, edits and users
  • Helpful users
    • button to click indicating spam, leading to any/all of:
      • add TODO item (which emails package admin and sysadmins)
      • quarantine the package/user so it doesn't show up on the front page and a warning is shown when you view it

General thoughts:

  • We should learn from the wikipedia tools, as they are the experts
  • This is a serious problem that is only going to get worse and is a threat to the whole site.
1312283565000000 1339774129000000
#1258 enhancement kindly kindly ckan-sprint-2011-10-28 closed fixed Bring purge revision into ckan repo from vdm

In order to make purge revision work correctly with the moderated edits we need to modify purge revision in vdm. This is best modified in ckan so we will override the vdm one in the reposotory.

1312289539000000 1319812452000000
#1259 enhancement johnglover pudo ckan-backlog new "Add a row" for Extras on Package form

The default package form offers 4 empty extras fields. Like the resource section, it should have an "add more" button to add another row.

1312302693000000 1312907056000000
#1260 enhancement pudo ckan-backlog new Remove duplicate functions from _util.html

There seems to be both a list view for dictized and non dictized data structures for all entities in _util.html at the moment. Probably in the back of someone's mind already, but cleanup here would be nice.

1312366652000000 1313401499000000
#1261 defect pudo ckan-backlog new Investigate dots in extras search

It seems that searching for extras_foo:value works with solr, but extras_foo.bar:value doesn't. No theory on why.

1312366768000000 1312366768000000
#1262 enhancement pudo ckan-backlog new Enforce "create-user" permission

This does not seem to have any implications at the moment, it should lock down registration and remove all related links.

1312375296000000 1323090112000000
#1263 enhancement dread dread closed fixed Bad name for logic action in API causes Exception

A bad action name: e.g. curl http://test.ckan.net/api/action/user_showaa -d '{}' causes 500 response.

Occurs only on 1.4.3 beta.

1312479020000000 1314029279000000
#1264 enhancement zephod rgrp ckan-sprint-2011-10-10 closed fixed Merge ckanext-admin into core

This extension should be in core now -- functionality is basic and very useful.

Copy across from here: http://bitbucket.org/okfn/ckanext-admin

Suggest this no longer even be an extension.

1312480783000000 1318245716000000
#1265 defect dread dread ckan-sprint-2011-10-28 closed fixed user/edit error conditions cause exceptions

We get exceptions on these occasions:

  • user/edit if not logged in
  • user/edit/bob if bob is not a known user

These are a problem only on 1.4.3b.

Introduced in #1229 - merge of branch feature-1229-db-out-of-controllers

1312565788000000 1312566052000000
#1266 defect dread dread ckan-sprint-2011-10-28 closed fixed Timestamps without microseconds causing exception

I'm not sure why some timestamps are missing microseconds in the ckan.net data, but plenty seem to. This causes this exception when viewing the package rss feed:

URL: http://ckan.net/package/history/sfk_publishing?format=atom&days=7
In Module ckan.lib.helpers:227 in date_str_to_datetime
WebApp Error: <type 'exceptions.ValueError'>: time data '2008-04-13T20:40:20' does not match format '%Y-%m-%dT%H:%M:%S.%f'

Only seen with ckan 1.4.3b

1312794831000000 1312797477000000
#1267 defect dread dread closed fixed Random exception: cannot set the body to a unicode value

Occasionally see this problem when producing an error page. Here's an example

WebApp Error: <type 'exceptions.TypeError'>: You cannot set the body to a unicode value without a charset
URL: http://at.ckan.net/authorizationgroup/new
Module weberror.errormiddleware:162 in __call__
<<              __traceback_supplement__ = Supplement, self, environ
                   sr_checker = ResponseStartChecker(start_response)
                   app_iter = self.application(environ, sr_checker)
                   return self.make_catching_iter(app_iter, environ, sr_checker)
               except:
>>  app_iter = self.application(environ, sr_checker)
Module beaker.middleware:73 in __call__
<<                                                     self.cache_manager)
               environ[self.environ_key] = self.cache_manager
               return self.app(environ, start_response)
>>  return self.app(environ, start_response)
Module beaker.middleware:152 in __call__
<<                          headers.append(('Set-cookie', cookie))
                   return start_response(status, headers, exc_info)
               return self.wrap_app(environ, session_start_response)
           
           def _get_session(self):
>>  return self.wrap_app(environ, session_start_response)
Module routes.middleware:130 in __call__
<<                  environ['SCRIPT_NAME'] = environ['SCRIPT_NAME'][:-1]
               
               response = self.app(environ, start_response)
               
               # Wrapped in try as in rare cases the attribute will be gone already
>>  response = self.app(environ, start_response)
Module pylons.wsgiapp:125 in __call__
<<          
               controller = self.resolve(environ, start_response)
               response = self.dispatch(controller, environ, start_response)
               
               if 'paste.testing_variables' in environ and hasattr(response,
>>  response = self.dispatch(controller, environ, start_response)
Module pylons.wsgiapp:324 in dispatch
<<          if log_debug:
                   log.debug("Calling controller class with WSGI interface")
               return controller(environ, start_response)
           
           def load_test_env(self, environ):
>>  return controller(environ, start_response)
Module ckan.lib.base:117 in __call__
<<          # available in environ['pylons.routes_dict']    
               try:
                   return WSGIController.__call__(self, environ, start_response)
               finally:
                   model.Session.remove()
>>  return WSGIController.__call__(self, environ, start_response)
Module pylons.controllers.core:284 in __call__
<<              if log_debug:
                       log.debug("Calling Response object to return WSGI data")
                   return response(environ, self.start_response)
               
               if log_debug:
>>  return response(environ, self.start_response)
Module webob.exc:248 in __call__
<<              return []
               if not self.body and not self.empty_body:
                   return self.generate_response(environ, start_response)
               return Response.__call__(self, environ, start_response)
>>  return self.generate_response(environ, start_response)
Module webob.exc:239 in generate_response
<<              status=self.status,
                   headerlist=headerlist,
                   content_type=content_type
               )
               return resp(environ, start_response)
>>  content_type=content_type
Module webob.response:94 in __init__
<<                  if charset is None:
                           raise TypeError(
                               "You cannot set the body to a unicode value without a charset")
                       body = body.encode(charset)
                   self._body = body
>>  "You cannot set the body to a unicode value without a charset")
TypeError: You cannot set the body to a unicode value without a charset
1312889084000000 1312897724000000
#1268 enhancement kindly rgrp ckan-sprint-2011-10-10 closed fixed Speed up listing users by revisions by adding db index

At the moment this appears to be slow (kindly suggests because missing index in db). Page is /user on a ckan instance.

Would this also speed up user page where we recent changes?

1312906592000000 1318279592000000
#1269 defect johnglover rgrp ckan-backlog closed wontfix User view page shows the packages followed by viewing user not user being viewed

This overlaps with Sean's follow support (for activity streams) and so will be dealt with there.

1312906598000000 1335875028000000
#1270 defect dread dread ckan-sprint-2011-10-28 closed fixed Default site www.ckan.net is a redirect and loses POSTs

The default site was www.ckan.net, which had an Apache 301 redirect to ckan.net, but the problem was urllib was not rePOSTing stuff. This affected search parameters:

In [1]: import ckanclient

In [2]: c = ckanclient.CkanClient()

In [3]: c2 = ckanclient.CkanClient('http://ckan.net/api')

In [4]: c.package_search('', search_options={'groups':'openspending'})
Out[4]: 
{u'count': 2102,
 u'results': <generator object _result_generator at 0x10168b1e0>}

In [5]: c2.package_search('', search_options={'groups':'openspending'})
Out[5]: {u'count': 29, u'results': <generator object _result_generator at 0x10168b410>}

Thanks to borior for finding this and raising it.

1312911847000000 1312912188000000
#1271 enhancement rgrp rgrp ckan-sprint-2011-10-28 closed fixed CORS support

CORS - http://www.w3.org/TR/cors/ - support.

This is what you do in Apache. Should do this in lib/base.py or similar.

    Header always set Access-Control-Allow-Origin "*"
    Header always set Access-Control-Allow-Methods "POST, PUT, GET, OPTIONS"
    Header always set Access-Control-Allow-Headers "X-CKAN-API-KEY, Content-Type"

    # Respond to all OPTIONS requests with 200 OK
    # This could be done in the webapp
    # This is need for pre-flighted requests (POSTs/PUTs)
    RewriteEngine On
    RewriteCond %{REQUEST_METHOD} OPTIONS
    RewriteRule ^(.*)$ $1 [R=200,L]
1313241839000000 1313753663000000
#1272 enhancement amercader ckan-sprint-2011-10-10 closed fixed Store spatial extents provided in the package form

Consolidate the extra spatial as the field for providing information about the geographic extent of the package. This will integrate automatically with the spatial search and give developers a single point to develop geospatial features.

1313411787000000 1317381996000000
#1273 requirement amercader ckan-backlog new Create docs for API v3 1313412083000000 1313412083000000
#1274 enhancement johnglover johnglover ckan-sprint-2011-10-28 closed fixed Testing solr search

Copy ckan core postgres package search tests to the ckanext-solr extension and update them so that they use the solr search backend.

1313413202000000 1313429049000000
#1275 enhancement johnglover johnglover ckan-sprint-2011-09-26 closed fixed Move solr search extension to ckan core 1313413597000000 1319812967000000
#1276 enhancement johnglover johnglover ckan-sprint-2011-09-12 closed fixed Apply weighting to search fields in solr

For example:

  • name and title should have a higher weight than the notes field
1313414353000000 1315948394000000
#1277 enhancement johnglover johnglover ckan-sprint-2011-09-12 closed fixed Use solr query parser for search instead of ckan query parser

See http://ckan.okfnpad.org/search for details/discussion

1313414446000000 1315948417000000
#1278 enhancement amercader ckan-backlog new Refactor authorized_query calls

There are some functions that still use the Auhtorizer().authorized_query method:

./ckan/controllers/authorization_group.py:24:        query = ckan.authz.Authorizer().authorized_query(c.user, model.AuthorizationGroup)
./ckan/lib/base.py:237:        groups = ckan.authz.Authorizer.authorized_query(c.user, model.Group, 
./ckan/lib/search/sql.py:55:        q = authz.Authorizer().authorized_query(username, model.Group)
./ckan/lib/search/sql.py:118:        q = authz.Authorizer().authorized_query(self.options.get('username'), model.Package)
./ckan/logic/action/get.py:154:    query = Authorizer().authorized_query(user, model.Group, model.Action.EDIT)

./ckan/tests/test_authz.py:158:        q = self.authorizer.authorized_query(self.notadmin.name, model.Package)
./ckan/tests/test_authz.py:353:        q = self.authorizer.authorized_query(self.notmember.name, model.Package)
./ckan/tests/test_authz.py:357:        q = self.authorizer.authorized_query(self.member.name, model.Package)
./ckan/tests/functional/test_authorization_group.py:44:        group_count = Authorizer.authorized_query(u'russianfan', model.AuthorizationGroup).count()
1313415177000000 1313415177000000
#1279 enhancement ross amercader ckan-v1.8 closed duplicate Refactor or deprecate Authorization Groups

Authorization Groups are going to be deprecated in 1.8

It is still unclear how Authorization Groups fit in the new authz model, so they have been left out of the logic and auth refactoring (Tickets #1229 and #1253)

1313415367000000 1338212028000000
#1280 defect kindly ckan-backlog closed fixed fix sqlalchemy so that it works with postgres 9.0

After the ckan migrate process, sqlalcehmy reflect does not work and causes an error when reflecting indexes. A bug report needs to be put into sqlalchemy.

1313513090000000 1328786670000000
#1281 defect pudo pudo ckan-sprint-2011-10-28 closed fixed Locale change flash message in wrong language

When you change the locale, it displays a flash message "Language has been set to: English" in the original language, not the one we have just changed to.

1314021656000000 1314021919000000
#1282 enhancement rgrp rgrp ckan-v1.5 closed fixed JS and WUI tidy up

Consolidate js. Remove cruft.

  • Move stuff to vendor
    • Have all vendor libs local (?)
    • jquery, jquery-ui etc
  • Consolidate autocomplete to use jquery-ui
    • package autocomplete (groups)
    • tag autocomplete (package)
    • user autocomplete (authz group create)
      • turns out that authz group edit had autocomplete disabled so refactoring meant enabling ...
  • Move forms.css into main
  • Remove tag cloud from front page (can be example going forward)
  • Remove bookmarklet (since in wiki at: http://wiki.ckan.net/Managing_Data_Packages#How_Do_I_Install_the_CKAN_Bookmarklet.3F)
  • Move name slug generation into standard js and clean up

Possible:

  1. JS unit tests for the given pages ...
  2. Option to use external site to serve js and css (e.g. CDN)

1314090389000000 1314404540000000
#1283 enhancement dread dread ckan-backlog closed fixed Deleted packages shouldn't be searchable or browsable

A package in deleted state doesn't show to general users, but admins DO see them. The idea is that they can resurrect them if they want. But now we have the trash can that lists deleted packages. So we don't want admins to see the deleted packages anywhere else.

(But an admin should still be able to READ and EDIT a deleted package, so if he clicks on it in the trash list he can see it to decide whether to resurrect and actually be able to change its state.)

Same in the Web interface and API.

This ticket ties in with #948 highlighting a package being deleted (when viewing or editing) and follows on from the introduction of the trash can #1076.

1314112481000000 1330083933000000
#1284 defect dread rgrp closed fixed Lots of revisions being created on datahub.org

Revisions aren't assigned to packages. Where are they coming from? This seems to have been occurring since the upgrade to 1.4.3b a couple of weeks ago?

1314113015000000 1315215626000000
#1285 enhancement dread ckan-future assigned Errors cause emails

Currently a sysadmin gets an email when an exception is not caught. But there are occasions when we DO want to catch an exception so we can fail nicely for the user, but the sysadmin STILL gets an email to know to fix something. e.g. if there is an exception when search indexes a package. You want to catch the exception and still run any other notify calls.

1314116944000000 1338206151000000
#1286 enhancement rgrp ckan-backlog new Remove remaining formalchemy stuff

Stuff I've spotted:

  • forms/*
  • template/group/edit_form.html
  • template/package/edit_form.html

This can go once new DGU form is in.

1314116996000000 1342436420000000
#1287 enhancement thejimmyg dread ckan-backlog assigned NAVL validation errors - Junk fields should be listed explicitly

When you create a package, but specify a key that is not allowed (e.g. 'relationships') then you get error message:

{"__junk": ["The input field __junk was not expected."]}

It should mention the actual key which is not expected. e.g.

{"relationships": ["The input field 'relationships' was not expected."]}

Kindly said that James' version of NAVL was better in this respect, so this might be best solved by moving to that.

1314203102000000 1330990459000000
#1288 defect dread ckan-backlog new Package edit/creation can't include 'relationships' field

When you create or edit a package (via the API), you aren't able to specify the relationships it has. (If you do you get 409 {"__junk": ["The input field __junk was not expected."]} )

The normal way to create relationships is via /api/rest/relationships/ and this works. But when you GET a package, the dictionary lists all relationship details. So this bug creates a problem for editing a package that has relationships - you want to GET it, make any edits and then PUT it back. The work-around is to delete the 'relationships' key from the dict before you PUT it back.

Options

Ideally, CKAN would read the 'relationships' key and make the necessary changes. This is a chunk of work.

Another good option is to allow an unchanged 'relationships' value, but barf it is edited. This is also a chunk of work.

A bad option would be to just ignore the 'relationships' value, since users will get frustrated changing this value and wonder why it never saves, not understanding it is different to all the rest, without error message.

A final option is to get rid of relationships altogether.

1314203695000000 1339774098000000
#1289 CREP dread ckan-backlog closed wontfix Remove 'relationships'

Abstract

Package Relationships have not taken off in the 18 months we've had them in the API. There are some issues with them and we need to spend more time improving them or consider getting rid of them.

The Problem

Original use cases are expressed here: #253 Here are comments about how we could handle these specific examples better:

  1. groups of packages - maybe better with a custom tag?
  2. fragment resources - soon to be covered by 'kind' resource field #957

3&5. derived resource - better to have some sort of resource relationship perhaps?

  1. linked resource - again better to have some sort of resource relationship perhaps?

Outstanding issues needing serious effort to fix:

  • #256 Editing them in Web UI (not done yet)
  • #1288 Package edit/creation can't include 'relationships' field

Specification

Remove relationships from model, API, tests, Web UI. Data migration to remove from db.

Why do it this way

Getting frustrated having problems with the code, when it's not used much. Often asked about what it's for, but rarely used. Seems an overly complicated design.

Backwards Compatibility

n/a

Implementation plan

Deliverables

See Specification

Risks and mitigations

Risk: a customer suddenly wants this, and the new ways to relate resources are not in place yet.

Mitigation: discuss this decision thoroughly to make sure we are confident the use cases are not important. Discuss with team, ckan-discuss and specifically the LOD people who have some related packages on thedatahub.org.

Participants

David Read

Progress

Not yet.

1314206502000000 1317315211000000
#1290 enhancement dread dread ckan-backlog closed fixed Better error when blank database

When installing CKAN, when doing "paster serve development.ini", lots of users encounter the error for every request:

ProgrammingError: (ProgrammingError) relation "user" does not exist

This is because the database tables have not been created - they have forgotten or missed the "paster db init" step.

Can we provide a better error to say that the database is not initialised yet?

Implementation options

  1. At the start of every request we reflect the database tables and check they are there. This is rather expensive!
  1. Uncached requests to the home page start with a cheap database query. If there is an exception then return this error about database setup. I really like Drupal's page for this that has in large letters that the site is currently off-line. Below a line, in small letters, there are developer suggestions on what is wrong and where to look to fix it.
1314264255000000 1314270894000000
#1291 enhancement kindly kindly ckan-sprint-2011-09-12 closed fixed Add activity and activity detail tables.

Add tables described in http://ckan.okfnpad.org/notifications

1314267572000000 1315950394000000
#1292 enhancement dread dread ckan-sprint-2011-11-07 closed fixed Internationalization improvements
  • Review process of releases and updating strings (pot) and translations (po) files
  • Get language changing thing to get home page to mostly work - push to key strings translated (or don't advertise the translation?)
  • Review long list of strings for improvement
    • Consolidate any?
    • Get rid of i18n for strings in api
    • Try and remove javascript
    • Add in extensions
  • #1374 Fix switching to en from other default language
  • #1417 Fix browser language detection
1314303314000000 1320145732000000
#1293 enhancement zephod rgrp ckan-sprint-2011-09-12 closed fixed Rename Package to Dataset throughout WUI, API

Changes

  • All templates
  • [DONE] Routing: /package/ -> /dataset/ (including API)
  • [DONE] Tests: self.app.get('/dataset/...') rather than package ...
    • One test to check redirect works?
  • Docs

Leave

  • Stay with Package in code e.g. stay with 'PackageController?'
    • also leave stuff like pkg = .... (gradual migration)
  • Leave extensions

Notes

  • Package -> Dataset
  • Data Package -> Dataset
  • respect capitalization
1314405700000000 1317649968000000
#1294 enhancement rgrp rgrp ckan-v1.5 closed fixed [super] Package creation and editing UX improvements

Largely an integration of work from ckanjs

  • #1295 Simplify package create form
  • Improvements to edit form
    • Split edit form into sections
    • Remove preview from edit form (introduce inline preview on description/notes field)
  • #1296 Improved resource adding/editing on dataset/page page
  • #878 File upload integrated into resource creation
  • #1297 In-place editing of notes field for existing packages
1314405811000000 1315948257000000
#1295 enhancement rgrp rgrp ckan-sprint-2011-09-12 closed fixed Simplify package create form

Sub ticket of #1294

  • Initial screen just title, name, description (notes), license (and tags?).
  • State that you can add data and more information on next screen. This takes you to Package 'view' where you can then do more stuff.
    • Flash some instructions? e.g. that resources are missing?
  • Name slug should by default look uneditable (cf wordpress way to do the slug)
1314406014000000 1315948009000000
#1296 enhancement zephod rgrp ckan-sprint-2011-09-12 closed fixed Improved resource adding/editing on dataset/page page

Subticket of #1294.

  • Nice way to edit all resource attributes
  • Nice way to add new resources
    • Including file upload (cf #878).
      • Cf work in ckanjs (this can be a straight integration).
1314406093000000 1315948224000000
#1297 enhancement zephod rgrp ckan-sprint-2011-09-12 closed fixed In-place preview of edited description markdown on edit dataset form

Subticket of #1294.

Can integrate from working system on ckanjs.

1314406183000000 1317649974000000
#1298 enhancement kindly kindly ckan-sprint-2012-01-09 closed fixed Generate activites to be put into activities table.

This should be done from the logic layer or automatically from a session extension.

1314696442000000 1325591582000000
#1299 enhancement seanh kindly ckan-sprint-2012-01-23 closed fixed Activity streams table migration

Migrate tables for activity streams

1314696635000000 1327322739000000
#1300 enhancement rgrp rgrp ckan-sprint-2011-11-07 closed fixed Core changes to base theme

Core changes to theme in order to make it easier to re-theme:

  • page_heading
  • side bar menu must be switchable (left to right) (through config or css)
  • Add optional_footer to complement optional_head
    • And adopt rule for template writers that all extra js must go in optional footer
    • That way we could move optional js to the bottom of the page to improve page load times.
  • Re-add support for body-class ... (seems to be removed by #1108)
  • more divs.
  • better labelling for css (and javascript)
1314862223000000 1319794520000000
#1301 defect amercader amercader ckan-sprint-2011-09-12 closed fixed Give access to extensions from the old formalchemy controllers

This was removed during the auth refactoring, but it's needed in the old controllers

1314882888000000 1314893744000000
#1302 refactor johnglover johnglover ckan-sprint-2011-09-26 closed fixed Move Resource and Tag search code to logic layer

Resource and Tag search is currently still using the old ckan query parser which is unnecessary. Refactor this out and move to the logic layer.

1314886102000000 1317131948000000
#1303 defect amercader amercader ckan-sprint-2011-09-12 closed fixed User can not login again if he has edited his profile

Only happens if the user does not update the password. Most probably introduced during #1229.

1314889472000000 1314978842000000
#1304 enhancement amercader amercader ckan-sprint-2011-09-12 closed fixed Show more descriptive error messages when sending emails

Sending emails is failing in some cases, and the error message just says 'Could not send reset link'.

1314893206000000 1314893749000000
#1305 defect nils.toedtmann amercader ckan-backlog closed fixed SMTP config for thedatahub.org and IATI

The email sending functionality (e.g for password reset) does not work on thedatahub.org and IATI (and probably some other instances) when using an address which is not a okfn.org one.

Could not send reset link: SMTPRecipientsRefused({u'[email protected]…': (550, 'relay not permitted')},)

As I said, [email protected]… works fine. The SMTP server used mail.okfn.org

1314956657000000 1315317033000000
#1306 enhancement kindly rgrp ckan-backlog closed wontfix Tests for the logic layer

AFAICT there are no tests for the logic layer at the moment. I imagine this is an issue (if it is not please explain and close as wontfix).

1315153267000000 1319797204000000
#1307 enhancement zephod rgrp ckan-sprint-2011-09-12 closed fixed Implement captcha for signup
  • Major issue with spam signup on thedatahub.org
  • Suggest using recaptcha

Est cost: 1d

1315153860000000 1317649963000000
#1308 defect shevski dread closed fixed 'create package' user case failure

On the CKAN home page there is no suggestion that you can add or register a dataset. The 'Register a dataset' option has been removed at some point! (only visible to users that have registered - why would they do that when the benefits aren't described?)

1315219500000000 1323174312000000
#1309 enhancement zephod zephod ckan-sprint-2011-09-12 closed duplicate Registering new user requires immediate login

Registering a new user directs you to that user's profile page but does not log you in. There is an empty set of "Recent changes" and no ability to modify the page. Users must follow the login link in the top-right corner to continue.

Registering a new user should immediately log you in.

1315225510000000 1315586335000000
#1310 defect dread dread ckan-sprint-2011-09-12 closed fixed API GET of a revision that has been approved causes exception

e.g. this particular revision on ckan.net: http://ckan.net/api/rest/revision/9601fbcb-03a7-4ebc-981d-5aa16df4852c (most others have not been approved) Causes 500 error.

1315234899000000 1315820419000000
#1311 enhancement rgrp rgrp ckan-backlog new Modal user register and login form

Subticket of: #1294

Rather than having to visit a dedicated page it would be good if registration and login could be done from a modal form (separate or combined ...).

Why

  • It could be used from dataset creation page in situations where user needs to be registered / logged in to create a dataset (so we could allow someone to start creating a dataset and only get them to login at the end ...)
  • It allows for quicker and easier logging in

Implementation

  • See Friedrich's work on the datahub
1315297227000000 1315297227000000
#1312 enhancement dread [email protected] ckan-sprint-2011-10-28 closed invalid Particular characters in JSON cause exception creating package on API

From Florian:

I am having trouble with the characters "=&;".
if they occur e.g. in the title of a package, I get a 400-Bad-Request
when I try to register a package by POSTing to ckan/rest/api/package

However, when I PUT the same package as an edit to
ckan/rest/api/package/existing_package, it works and the title shows
correctly on the front end and API.

Reproduced

(pyenv-ckan)[email protected]:~/hgroot/ckan$ curl -i http://localhost:5000/api/rest/package -d '{"name": "test3", "title": "Test &"}' -H "Authorization:tester" 
HTTP/1.0 400 Bad Request
Server: PasteWSGIServer/0.5 Python/2.6.5
Date: Tue, 06 Sep 2011 14:09:31 GMT
Pragma: no-cache
Cache-Control: no-cache
Content-Type: application/json;charset=utf-8
Content-Length: 87

"Bad request - JSON Error: Unterminated string starting at: line 1 column 27 (char 27)"
1315318863000000 1315327743000000
#1313 enhancement zephod dread closed duplicate Tweaks to captcha configuration

Awesome to have the captcha implementation and running on thedatahub.org! Here are some further suggestions concerning the config options, before this goes into a release:

  • Document "ckan.recaptcha.publickey" and "ckan.recaptcha.privatekey" in doc/configuration.rst
  • Which one config option defines whether recaptcha is enabled? There is disagreement between the template and the lib. Maybe better to have a third option for this. When setting the syntax for this, I can see us wanting to use captcha for other operations, like package edit, package create, group create etc. so perhaps we'd want to be able to turn each one on/off separately, using the name of the logic action. e.g. ckan.recaptcha.enable = user_create, group_create
  • We got this exception last night from the template:
    Module ?:37 in <Expression u'g.recaptcha_publickey'>
    <<          <dd><input type="password" name="password2" value="" /></dd>
           
                   <dd py:if="g.recaptcha_publickey">
                     <script type="text/javascript"
                       src="http://www.google.com/recaptcha/api/challenge?k=${g.recaptcha_publickey}">
    >>  <dd py:if="g.recaptcha_publickey">
    Module genshi.template.eval:324 in lookup_attr
    <<                      val = obj[key]
                           except (KeyError, TypeError):
                               val = cls.undefined(key, owner=obj)
                   return val
    >>  val = cls.undefined(key, owner=obj)
    Module genshi.template.eval:410 in undefined
    <<          """Raise an ``UndefinedError`` immediately."""
                   __traceback_hide__ = True
                   raise UndefinedError(key, owner=owner)
    >>  raise UndefinedError(key, owner=owner)
    UndefinedError: <ckan.lib.app_globals.Globals object at 0xb8f1182c> has no member named "recaptcha_publickey"
    

I'm not sure why - might be something to do with the maintenance on the server, but worth double checking, since we don't want exceptions, say if one or other config variable is not defined.

1315395185000000 1317076432000000
#1314 enhancement dread ckan-backlog assigned ckanclient search - generator improvements

Apparently the search generator always makes two requests, even if you don't want to see the search results, which might be slow. Can this be optimised?

Maybe we should also provide a second search function that doesn't use the generator - the original simple search function (that leaves the user to deal with limit & offset).

1315395410000000 1340191233000000
#1315 enhancement dread dread ckan-sprint-2011-11-07 closed fixed Document /api/action

We need some documentation for the /api/action. Even though it's beta and subject to change it would be good to have a guide.

1315396722000000 1320173768000000
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Note: See TracReports for help on using and creating reports.