CKAN: Ticket Query

#2622: Login fails in Opera 12

seanh — Thu, 28 Jun 2012 16:56:42 GMT

Try to login to CKAN using Opera 12, get "Login failed. Bad username or password. (Or if using OpenID, it hasn't been associated with a user account.)"

#2683: Add no-cache header to _tracking API call's response to make sure it doesn't get cached

seanh — Mon, 16 Jul 2012 13:49:37 GMT

#3020: Update CKAN coding standards

seanh — Fri, 16 Nov 2012 20:11:51 GMT

#2763: Multilingual tests failing

seanh — Thu, 26 Jul 2012 11:56:59 GMT

test_multilingual_plugin.TestDatasetTermTranslation?.test_dataset_index_translation, test_multilingual_plugin.TestDatasetTermTranslation?.test_group_read_translation both failing for me on master

#2815: db_to_form_package_schema() strips tracking summary, isopen

seanh — Wed, 08 Aug 2012 16:47:07 GMT

If an IDatasetForm plugin with a db_to_form_schema() based on db_to_form_package_schema() (which is in turn based on default_package_schema()) is in use then the 'tracking_summary' dict and the 'isopen' bool get stripped from package dicts during validation, e.g. during package_show(), and these values are then not available to templates.

#2846: Organizations allows you to set an organization as the parent organization of itself

seanh — Mon, 13 Aug 2012 12:22:16 GMT

that doesn't make any sense

#2881: Remove or improve 'Domain Model' docs page

seanh — Tue, 21 Aug 2012 09:05:17 GMT

http://docs.ckan.org/en/ckan-1.8/domain-model.html

Currently not very useful. Also these pages:

http://docs.ckan.org/en/latest/domain-model-dataset.html

http://docs.ckan.org/en/latest/domain-model-resource.html

http://docs.ckan.org/en/latest/domain-model-task-status.html

This just seems very incomplete to me (surely there is more to the domain model the dataset, resource and task status?) and very likely to not get updated as changes are mode to the code. Would anyone seriously refer to this rather than just looking at the source code?

If we want to have something along these lines in the docs, I suggest adding docstrings to packages, modules, classes and methods and then using autodoc to pull them into sphinx.

#2882: Remove User Stories Overview from docs?

seanh — Tue, 21 Aug 2012 09:18:43 GMT

Why is this here? Is it up to date? Is anyone keeping it up to date? Does this belong in some sort of CKAN planning or roadmap document rather than in the documentation?

http://docs.ckan.org/en/latest/user-stories.html

#2883: Add high level dev overview to 'For CKAN Developers' section of docs

seanh — Tue, 21 Aug 2012 10:52:35 GMT

Here's a draft: https://gist.github.com/3414107

#2884: Lock icon shown next to unknown licenses

seanh — Tue, 21 Aug 2012 16:01:13 GMT

Set the license of a dataset to blank (the first option) or to 'License Not Specified' or 'Other (Non-Commercial)' and a lock icon appears next to the license on the dataset read page. Shouldn't this be a question mark or something?

#2887: "Welcome to CKAN!" on front page is untranslatable

seanh — Wed, 22 Aug 2012 09:17:59 GMT

ckan/templates/home/index.html contains:

<h1 class="page_heading">Welcome to ${g.site_title}!</h1>

which means that just "Welcome to" ends up in the ckan.po files for translation. This makes the phrase untranslatable in languages that have a different word ordering than English where the site title needs to be somewhere other than at the end of the sentence.

#2890: Collect data previews and data store docs in one chapter

seanh — Tue, 28 Aug 2012 10:20:36 GMT

Currently there is this page:

http://docs.ckan.org/en/latest/data-viewer.html

which covers Recline Data Explorer and other kinds of data preview in CKAN. It is under the Publishing Datasets section in the documentation. I had to to a search for 'recline' to find it.

Separately there is this page: http://docs.ckan.org/en/ckan-1.7.1/datastore.html which covers datastore, datastorer, and the data api.

I suggest collecting this together in one chapter called 'Data Previews'. If I understand it right the general gist would be:

CKAN has builtin previews of data resources on resource pages, enabled by default.

Images, Google Documents, and web page resources will be loaded into embedded iframes for preview.

Text-like files will be displayed raw.

CSV or Excel files uploaded to CKAN will be previewed using Recline Data Explorer.

Additionally, you can enable CKAN's DataStore?, requires you to install ElasticSearch? and nginx and put datastore.enabled=1 in your ini file. Lets you use the Data API to query data.

Does having DataStore? enabled mean you get preview of more types of resources? Any resource that's available via the Data API will be previewed using Recline,

You can install ckanext-datastorer, and then CSV and Excel files _linked to_ as CKAN resources will be previewed using Recline also. Requires celeryd.

#2891: Update or remove DataStore google presentation

seanh — Tue, 28 Aug 2012 10:23:43 GMT

Currently this page:

http://docs.ckan.org/en/ckan-1.8/datastore.html

has an embedded Google docs presentation about data store which contains some outdated information, e.g. links to datastoreclient which was merged into ckanclient.

Suggest moving the google presentation into the documentation proper, so it works with grep etc.

#2892: IDatasetForm's form_to_db_schema() breaks API?

seanh — Tue, 28 Aug 2012 11:42:05 GMT

It looks like when an IDatasetForm plugin with a form_to_db_schema() method is active, then the posted data dict is validated against the schema even if the user is posting to an API endpoint like /api/action/package_update etc. This breaks the API, for example 'name' and 'title' are optional keys for package_update (you can supply only the 'id'), but the schema validation will fail if either are not present in the posted dict.

It may be we need different schema for package create and package update, currently IDatasetForm has just the one form_to_db_schema() method.

#2893: Tags missing from form_to_db_package_schema()

seanh — Tue, 28 Aug 2012 12:11:04 GMT

With an IDatasetForm plugin active with a form_to_db_package_schema() method that returns a schema based on ckan/logic/schema.py:form_to_db_package_schema() I'm getting validation errors like this:

[ckan.controllers.api] Validation error: "{'type': 'Validation Error', 'junk': [u'The input field junk was not expected.']}"

They seem to be caused when a data dict with key 'tags' is passed to e.g. the package_update API, for example in ckan/tests/functional/api/model/test_vocabulary.py there are various tests that get this error when they call package_update (if you run the tests with a suitable IDatasetForm plugin active).

In ckan/logic/schema.py:form_to_db_package_schema() 'tags' is for some reason deliberately removed from the schema.

#2894: Empty lists get stripped from package dicst in package_show()

seanh — Tue, 28 Aug 2012 12:40:34 GMT

If there's an IDatasetForm plugin active with a form_to_db_schema(), then packages with e.g. no extras get the 'extras' key entirely stripped by the validation in package_show, which causes some tests to crash, e.g. in ckan/tests/functional/api/test_activity.py (if the tests are run with a suitable IDatasetForm plugin active).

If a package has no extras the dict returned by package_show should still have an 'extras' key with an empty list as the value.

Suspect this also effects packages with no tags, no groups, etc.

#2896: resource_create crashes on ValidationErrors with KeyError: 'resources'

seanh — Wed, 29 Aug 2012 16:29:14 GMT

ckan/logic/action/create.py:resource_create():

try:

pkg_dict = _get_action('package_update')(context, pkg_dict)

except ValidationError?, e:

errors = e.error_dictresources?[-1] raise ValidationError?(errors)

This assumes that package_update always returns an error dict with 'resources' in it. This assumption doesn't hold.

#2897: db_to_form_package_schema() crashes historical dataset page

seanh — Thu, 30 Aug 2012 12:41:35 GMT

If there's an IDatasetForm plugin active with a db_to_form_schema() method based on db_to_form_package_schema(), then viewing historical versions of datasets (e.g. http://127.0.0.1:5000/dataset/dasd%402012-08-30T12%3A26%3A18.524191) crashes.

Reason is that revison_id and revision_timestamp need to be added to db_to_form_package_schema(), something like:

schema.update({'revision_id': [validators.ignore_missing, unicode]}) schema.update({'revision_timestamp':

[validators.ignore_missing, unicode]})

#2900: snippets/package_list.html template crashes on dataset that has no 'notes'

seanh — Thu, 30 Aug 2012 14:00:07 GMT

Sometimes a dataset dict comes through package_show() with 'notes': None and then the 'notes' key gets stripped from the dict when validation happens (validation only happens if there's an IDatasetForm plugin with a db_to_form_schema() method) and then package_list.html template crashes, e.g. on user profile pages.

Fix in the template? Or in the schema?

The template also crashes on packages with no tags (empty list gets stripped from dict) and probably other empty fields as well

#2912: ckanext/organizations: Neither public or private initially selected when adding dataset

seanh — Mon, 10 Sep 2012 09:57:56 GMT

When adding a dataset with the organizations extension on, neither the Public or the Private radio button is initially selected. Public should be selected.

#2914: Add form_to_db_schema_options() to IDatasetForm and IGroupForm

seanh — Wed, 12 Sep 2012 10:29:55 GMT

This optional method is supported when creating or updating groups and packages and is used in DefaultGroupForm? and DefaultPackageForm?, but seems to be missing from the interfaces. Should be added with docstring.

#3006: Update translations from Transifex

seanh — Mon, 29 Oct 2012 10:22:24 GMT

There are some new translations already on the CKAN 1.8 resource on Transifex already (e.g. 100% Norwegian translation I think). We should at least pull these from Transifex for 1.8.1, and may as well do a call for translations too.

#1667: Add an extension point for rendering activity streams

seanh — Tue, 17 Jan 2012 10:29:11 GMT

Currently rendering of activity stream events to HTML works by looking up a rendering function in a dictionary that maps activity types ('new package', 'changed group', etc.) to rendering functions that take an activity stream event and return the rendered HTML.

There needs to be an extension point where extensions can register their own rendering functions for particular activity types.

#1740: Get rid of `from module import ...`

seanh — Wed, 01 Feb 2012 11:06:09 GMT

It's really bad to do from module import * and CKAN has a lot of them. I suggest a three-pronged approach:

Don't add any more of them.

When you're programming if you see an easy opportunity to remove one then do so.

At some point we should task someone to go through the code and remove them all (which is what this ticket is for), but this will be a big job and may break things.

We should also get rid of most or all of the from module import foo and from module import foo, bar statements.

I think the right thing to do is just import module and then use module.foo in your code. But if you find yourself doing module.foo.bar then you may have a code smell.

See: http://docs.python.org/dev/howto/doanddont.html

#1748: Make activity streams conform to http://activitystrea.ms/ standard

seanh — Sun, 05 Feb 2012 14:06:24 GMT

At the very least we should make our JSON output conform to their spec:

http://activitystrea.ms/

#1749: Allow creating activity details through API

seanh — Sun, 05 Feb 2012 18:10:06 GMT

Currently the activity_create() logic action function only lets you create top-level activity stream items, and not their related activity details. It should handle activity details via nested dicts.

#1778: Replace classmethod's with module-level functions

seanh — Mon, 06 Feb 2012 15:53:34 GMT

In many places we have @classmethods where simple module-level functions would do (and would be more idiomatic in Python), e.g. the get() classmethods in the model/ classes.

Doing it with module functions lets us type module.function() instead of module.Class.method().

In many cases we're importing classes directly with from module import Class and then doing Class.method(), but from module import foo is bad style: http://docs.python.org/dev/howto/doanddont.html

#1800: Tidy up _list() and _search() functions in ckan/logic/action/get.py

seanh — Thu, 16 Feb 2012 15:12:09 GMT

For consistency all the *_list() functions should list objects only and not accept an optional search query. There should be *_search() functions whenever search is needed.

Currently it is inconsistent, e.g. package_list() and resource_list() do not accept a search query option and there are package_search() and resource_search(), but user_list() does accept a query and there is no user_search(). tag_list() also accepts a query, and there is also a tag_search() that apparently duplicates the search functionality.

#1824: Add vocabulary pages

seanh — Tue, 21 Feb 2012 17:24:49 GMT

For a free tag foo you can visit the page at /tag/foo and see a list of all the datasets that have the tag foo, and when the tag appears on dataset view pages etc. it's linked to this tag page.

We should do the same thing for vocabulary tags. A tag bar in vocabulary baz should be hyperlinked to a page /tag/baz/bar, or perhaps /vocab/baz/bar.

#2200: Add vocabulary_id option to tag_show() logic action function

seanh — Wed, 29 Feb 2012 16:41:18 GMT

Currently tag_show() only works with free tags, it's not possible to get a tag_show for a vocabulary tag.

#2234: Write a CKAN extension for pulling items from RSS/Atom feeds into CKAN templates

seanh — Fri, 16 Mar 2012 12:59:15 GMT

You configure the extension with some RSS and/or Atom feeds, it automatically reads items from these feeds and makes them available in the template context, you write a custom template to e.g. display 'news' items from a Wordpress blog on your front page.

This extension might be simpler and less fragile than ckanext-wordpresser, and also more generally useful.

Details

Mark Pilgrim's Universal Feed Parser might be useful for reading the feeds

Feed items should probably be cached somewhere

Suggested

The news item 'widget' should be wrapped in a known class so that it can be styled easily regardless of the format of any HTML entry.

For non-HTML formatted items (Atom should tell you the content type of the entry) maybe we should have a template for rendering each item along with any enclosures that it might reference

Caching is pretty crucial and should probably obey the ttl of the feed.

#2287: Consistent datetime and time period strings localization

seanh — Thu, 12 Apr 2012 18:08:43 GMT

Find all places where date, times and time periods are rendered as strings in CKAN. Make them all go through the same code path (e.g. the helper function in helpers.py) with localization support.

For example, on the user index page right now there are time period strings like "less than 1 month" that currently do not get translated (and the way they're implemented doesn't look i18n-friendly).

ISO date format everywhere might be a good idea.

Need to consider local timezone issues.

Python standard library and Babel may have useful helpers for this.

Should the local timezone displayed be a global setting per CKAN instance? Or should it change depending on the location of the user who is viewing the page?

#2301: Replace old-style string formatting with format() method (at least in strings marked for translation)

seanh — Mon, 16 Apr 2012 12:27:44 GMT

Some strings in CKAN are passed to gettext with more than one %s in them, this is no good as translators may need to change the order of substituted words.

Strings should not use the old style % substitution, they should use the new format method in Python 2.6+, e.g.:

'This {food} is {adjective}.'.format(food='spam', adjective='absolutely horrible')

http://docs.python.org/release/2.6.8/tutorial/inputoutput.html#fancier-output-formatting

We should probably only change strings that are marked for translation and that contain multiple %s, don't want to make too much work or cause too many strings to change and need to be re-translated.

#1635: Email notifications (e.g. for activity streams)

seanh — Wed, 11 Jan 2012 17:56:27 GMT

CKAN should be able to send email notifications to users.

Maybe have a notifications table in the db, and a server-side job that runs periodically and consumes rows from this table, mailing them to the users.

One thing that we may want to send users notifications of is activity stream events. So the activity streams code would have to add rows to the notifications table for the mailer job to consume. But remember that email notifications feature is separate from activity streams - we may want to send notifications of other things as well.

Need to implement (at least some of) #1634 before this can be implemented, in order to have something to send notifications about.

Analysis here: http://ckan.okfnpad.org/27

#1747: Expire old activities

seanh — Sun, 05 Feb 2012 12:56:29 GMT

Currently the activity streams database tables just get longer and longer over time. Do we want to eventually delete the oldest activities, to keep the length of the table within limits?

#1750: Move ckan/lib/activity.py into the model

seanh — Sun, 05 Feb 2012 18:18:08 GMT

Move ckan/lib/activity.py moved to into the model - say ckan/model/activity_extension.py, because it's so tightly knit with the model code, whereas most of the lib code is used in the controllers.

#1789: Implement a tag_update() logic action function

seanh — Thu, 09 Feb 2012 16:36:53 GMT

So users can rename a tag and/or move it between vocabularies using the API.

Currently we have create_tag() and delete_tag(), but if you were to 'update' a tag by deleting it and then recreating it all the datasets that had that tag will have lost it and you'll have to re-add it to them all.

What should happen to datasets that have the tag, if the tag gets moved between vocabularies? All the datasets just keep the tag with the new vocabulary? This will become a problem if/when we support 'radio button'-style vocabularies (where each dataset can only have one tag from the vocabulary).

#2486: Should be able to use . in dataset names

seanh — Sat, 02 Jun 2012 16:46:23 GMT

#2508: Make it possible to run CKAN tests for each language

seanh — Mon, 11 Jun 2012 10:42:15 GMT

Mistakes in translated strings can cause CKAN to crash or otherwise not work, but it's not practical to manually test every page and function of CKAN in every language that we have new translations for before a CKAN release. It'd be great if the tests could automatically be run for each language.

This is probably a big job, we would have to get the tests to respect a language setting in the ini file, check for any individual test cases that specify the language (e.g. in the URL), and also fix test cases that look for specific English words in HTML output, etc.

In the meantime, a good stop-gap solution might be a script that tests for common mistakes in the po files.

#2607: 'Upload a file' appears on resource form when storage not enabled

seanh — Wed, 27 Jun 2012 13:30:08 GMT

if the user tries to upload a file they will get "Failed to get credentials for storage upload. Upload cannot proceed"

Maybe add a test for it this time, this bug has appeared and reappeared before

#2913: IGroupForm has package_form() method

seanh — Tue, 11 Sep 2012 14:45:39 GMT

Should be group_form() I think

#2915: Refactor form_to_db_schema_options()

seanh — Wed, 12 Sep 2012 10:37:49 GMT

Having two methods form_to_db_schema(self) and form_to_db_schema_options(self, options) seems unnecessary, why not just have form_to_db_schema(self, options=None)?

Fixing this might break existing extensions although I don't think any are using form_to_db_schema_options() so it should be okay.

The same thing goes for db_to_form_schema_options().

Also why are we passing a dict 'options' that always contains the same three keys 'api', 'context' and 'type'? Why not three params api=None, contenxt=None, type=None?

#2920: Genshi text template message extractor no longer needed?

seanh — Thu, 13 Sep 2012 09:57:29 GMT

In setup.py, do we still need:

('templates/.txt', 'genshi', {

'template_class': 'genshi.template:TextTemplate'

}),

in message_extractors? Doesn't like there are any txt files in there anymore.

#2921: Add docstring to top of lib/extract.py file

seanh — Thu, 13 Sep 2012 10:00:07 GMT

I think it couldn't hurt for this module to have a docstring at the top of the file explaining what the module is for. I know from setup.py that it's there to provide the extract_ckan() string extractor function for Babel, but I think looking at the file on its own it's not so obvious.

Also a couple of other small fixes:

jinja2_cleaner looks like a helper function only meant to be used by extract_ckan(), might make things clearer if it was called _jinja2_cleaner(), just so it doesn't look like a public function the module wants to export.

jinja_extensions seems to be a module-level variable that is only used in one function, could be moved into the function, unless you think there might be more functions that want it in future.

#2922: Better docstring for CKANInternationalizationExtension

seanh — Thu, 13 Sep 2012 10:01:47 GMT

I'm unsure about what's going on here. As I understand it, when we run python setup.py extract_messages it's going to use the extract_ckan() function from ckan/lib/extract.py to process the HTML files. For the HTML files that are jinja2 templates, extract_ckan() will call jinja2_cleaner() which will call regularise_html() on the strings. So the strings are regularised when they are extracted from the source files.

But then we have the parse() method of CkanInternationalizationExtension? also calling regularise_html(). I don't get what's happening here. Why do the strings need to be regularised twice?

My guess is that CkanInternationalizationExtension? is used when the strings are extracted from the templates at runtime, and they need to be regularised at this time in order to match them against the regularised strings in the mo files to find the translations to output?

Maybe CkanInternationalizationExtension? needs a better docstring saying what it does?

#2923: Change regularise -> regularize

seanh — Thu, 13 Sep 2012 10:03:02 GMT

The function is called regularise_html(), can't remember what file it's in.

#2924: Better docs for trans js command, and add to release process

seanh — Thu, 13 Sep 2012 10:04:31 GMT

Add a better docstring to for trans js command explaining what it does and why, and how to use it.

Also add this command to the CKAN Release Process as it's needed there

#2925: Remove trans mangle paster command?

seanh — Thu, 13 Sep 2012 10:06:08 GMT

Is trans mangle really necessary? If you upload a pot file to Transifex, it can generate a po file for you with 100% strings translated into a fictional pseudo language where everything is really long strings of unicode characters. I found this worked well for coverage testing, and also tests handling of unicode and long strings all over the place.

#2926: I*Form tests for per-type templates

seanh — Thu, 13 Sep 2012 16:01:21 GMT

Finish the tests for the per-package-type custom template support in IPackageForm, IGroupForm and IOrganizationForm (i.e. the new_template(), read_template() etc. methods).

The new way that template inheritance works in jinja looks like it should allow this to be tested nicely without duplicating entire templates in the example extensions.

#2927: Test new package/group/organization type URLs support

seanh — Thu, 13 Sep 2012 16:02:33 GMT

IPackageForm, IGroupForm and IOrganizationForm allow you to define e.g. new package types with new top-level URLs, but the example extensions don't have tests for this yet.

#2928: Run CKAN tests with example_i*form extensions enabled

seanh — Thu, 13 Sep 2012 16:05:34 GMT

Before releasing CKAN 2.0 we need to run all the CKAN tests with a modified test-core.ini with the example_idatasetform, example_igroupform and example_iorganizationform plugins enabled. If any tests fail, fix the bugs. This needs to be done for each release so add it to the release process.

#2929: Remove is_fallback() from I*Form extension interfaces?

seanh — Thu, 13 Sep 2012 16:09:57 GMT

If a plugin wants to take over the default package type for example, it can return dataset? from its package_types() method. I don't see how there can be a package type that is not the default 'package' and is not returned by the package_types() method of a plugin, because the URLs for such a package type would not be mapped. Am I missing something or is is_fallback() not needed?

Same for IGroupForm and IOrganizationForm

#2931: Better docstring for app_globals.py

seanh — Mon, 17 Sep 2012 14:13:14 GMT

The application's Globals object is not very informative.

#2932: Add docstring to system_info.py

seanh — Mon, 17 Sep 2012 14:50:29 GMT

What is the system_info table for? It looks like a way to override config file settings in the database, perhaps so that admins can use a web interface to change site title etc. Could do with a docstring explaining the intended purpose of the table.

#2933: Update theming docs

seanh — Mon, 17 Sep 2012 15:21:12 GMT

There's quite a lot of custom stuff going on, with all the custom jinja extensions, and the use of Fanstatic plus custom fanstatic extensions. People who are new to CKAN wanting to write CKAN themes are likely to be confused. I think the whole lot could do with documentation, maybe in the Theming and Customizing chapter in the sphinx docs.

I suggest completely replacing the contents of doc/theming.rst, and putting a 'legacy theming' link at the bottom linking to a doc/legacy_theming.rst file with the old contents.

#2964: Last organization admin can remove herself

seanh — Mon, 15 Oct 2012 10:14:18 GMT

If you are the only admin of an organization you can edit the organization's members and demote yourself, then the org has no admins and no one (except sysadmins maybe) can edit it.

Last admin should not be able to remove or demote herself.

Also applies to groups.

#2966: 'Add' button text is wrong when editing organization members

seanh — Mon, 15 Oct 2012 10:15:49 GMT

e.g. if I just changed a member's capacity I am not adding anything 'save' is better maybe

#2967: Organization members edit page reloads after demoting self

seanh — Mon, 15 Oct 2012 10:17:07 GMT

Edit an organizations members page and demote yourself from organization admin, after saving the members edit page reloads even though you are no longer an admin and should no longer be able to access this page.

#2968: Anyone can access organization members page

seanh — Mon, 15 Oct 2012 10:19:15 GMT

The button will not show if you are not authorized but browse to /organization/members/foo and you can edit the members, it does stop you when you try to save your changes, but you shouldn't be able to get to the page at all

#2969: Group members page 500s

seanh — Mon, 15 Oct 2012 10:20:54 GMT

The group members page (e.g. /group/members/roger) seems broken, IDs instead of user names are shown for the members, and clicking on a member 500s

#2970: Organization and group member links use id not name

seanh — Mon, 15 Oct 2012 10:22:11 GMT

e.g. it the 'Members' button links to /organization/members/0a44... instead of /organization/members/foobar

#2971: "Are ytou sure you want to delete this member?" should say which member

seanh — Mon, 15 Oct 2012 10:27:20 GMT

when deleting members from groups and orgs

#2972: Remove any imports of authz.py and delete file

seanh — Mon, 15 Oct 2012 10:37:22 GMT

it is no longer used

#2973: Move new_authz.py into logic/auth/init.py

seanh — Mon, 15 Oct 2012 10:39:05 GMT

Makes sense to keep all auth stuff under logic/auth.

Also decide which functions from new_authz.pu should be marked private with _.

Also decide which functions from new_authz.py are really helpers for the auth functions in get.py, update.py and delete.py, and should maybe moved to another file, e.g. logic/auth/helpers.py

#2974: General of all auth functions

seanh — Mon, 15 Oct 2012 10:40:12 GMT

From Toby:

A general cleanup of all auth functions (in logic/auth) to check their fitness, error messages, etc.

#2975: Tests for auth functions and new actions

seanh — Mon, 15 Oct 2012 10:41:00 GMT

From Toby: tests around the auth functions and new actions - probably needs an improved testing mechanism and things like test data creation on via actions

#2976: Polish group and organization member pages

seanh — Mon, 15 Oct 2012 10:41:49 GMT

From Toby: A little polish to the member pages and probably a little hardening of the controller and logic actions

#2977: Fix user autocomplete on group and organization member pages

seanh — Mon, 15 Oct 2012 10:42:29 GMT

From Toby: @johnmartin the user autocomplete on member add needs fixing it gets data but does not understand what to do with it

#2978: Tests for permissions for organizations and groups

seanh — Mon, 15 Oct 2012 10:43:26 GMT

From Toby: checking the logic around the who can do what from the user stories - via some tests may be a good approach

#2979: Requesting membership to groups and organizations

seanh — Mon, 15 Oct 2012 10:44:04 GMT

From Toby: The requesting membership user stories and implementation remain needed

#2981: Remove config from jinja templates

seanh — Mon, 15 Oct 2012 10:46:00 GMT

From Toby: remove the config from jinja2 templates get all such stuff via g. (before 2.0)

#2982: Move functionality from controllers into template helpers

seanh — Mon, 15 Oct 2012 10:46:42 GMT

From Toby: strip more functionality from controllers and add via template helpers eg activity streams

#2983: Refactor lib/base.py to remove circular import issues

seanh — Mon, 15 Oct 2012 10:47:24 GMT

From Toby: c) refactor lib/base.py to remove the circular import issues (render functions to lib/render.py - would fix much of that)

#2984: Auto-populate context with user, model, session, etc.

seanh — Mon, 15 Oct 2012 10:48:21 GMT

From Toby: make the context auto-populated with things like user/model/session if not user supplied - especially for extensions (pre 2.0) maybe via a helper or else a decorator on the action - helper seems nicer maybe done via get_action()

#2985: Make lib/helpers.py more template-specific

seanh — Mon, 15 Oct 2012 10:49:48 GMT

From Toby: make lib/helpers more template specific and less generally shared with none template uses (some functions are shared but many shouldn't be) - again help avoid circular imports

#2986: Make lib/base.py more like it was originally intended

seanh — Mon, 15 Oct 2012 10:50:45 GMT

From Toby: make lib/base.py be more like it was originally intended eg defines stuff that is shared eg _ but maybe try to include stuff like render/get_action if the circular import stuff can be solved - which should be doable if hard and likely needs an extra file or two for the externals _ etc

#2987: Remove all direct calls to logic action and auth functions

seanh — Mon, 15 Oct 2012 10:51:27 GMT

From Toby

#2988: UI functional tests for CKAN 2.0

seanh — Mon, 15 Oct 2012 11:02:15 GMT

Afaik the new CKAN 2.0 frontend has no functional tests

#2989: "Add dataset to organization" should auto-select the organization

seanh — Mon, 15 Oct 2012 11:04:12 GMT

'Add dataset to organization' button, when you get through to the third stage of the new dataset form the organization you came from is not selected.

I don't see any option to choose the group when adding or updating a dataset, but if I add a dataset via the "Add dataset to group" button on a group's page, then the dataset seems to get added to that group. (And I can also add/remove existing datasets by editing the group.) I wonder if organizations should work the same way, instead of having an Organization drop-down when creating or updating a dataset. The add dataset page needs to somehow indicate that you're adding a dataset to a certain group or organization though, doesn't currently.

#2990: Fix descriptions of groups and organizations on /groups and /organizations pages

seanh — Mon, 15 Oct 2012 11:04:51 GMT

#2991: Add blocks to header.html template

seanh — Tue, 16 Oct 2012 11:10:20 GMT

Currently if you want to customise the header.html template you have to copy the whole thing, it isn't broken up into blocks like other templates are.

#2993: "logged_in" and "visitor" show in user list at /users

seanh — Wed, 17 Oct 2012 09:42:02 GMT

#2994: Add dataset and user popovers throughout the site

seanh — Wed, 17 Oct 2012 14:26:04 GMT

If you hover the mouse over a dataset or user in an activity stream, you now get a popover with some info about that dataset or user and a follow/unfollow button.

These popovers should appear wherever datasets or users are listed, throughout the site.

#2995: Popovers for resources, groups, organizations, tags...

seanh — Wed, 17 Oct 2012 14:27:21 GMT

If you hover the mouse over a dataset or user in an activity stream, you now get a popover with some info about that dataset or user and a follow/unfollow button.

It would be nice to add similar popovers for other types of object in CKAN, e.g. resources, groups, organizations, tags...

#2996: Fix behaviour of View Profile button in user popover when already on user's profile

seanh — Wed, 17 Oct 2012 14:30:38 GMT

The user popovers that appear when you hover the mouse over a user in an activity stream include a View Profile button that takes you to that user's profile page. If you are already on that user's profile page then it simply reloads the same page, but it sends you to the Datasets tab if you were on another tab.

Possible solutions:

Don't show button if already on user's page?

Button just makes popover disappear, if already on user's page?

Button does reload the page, but reloads the same tab (datasets, followers, activity stream) that you were on.

Marking this low priority

#2997: Add activity streams to dataset pages

seanh — Wed, 17 Oct 2012 14:31:46 GMT

Add an activity stream tab to dataset pages, like we have on user profile pages. Dataset activity streams are already implemented in the backend.

#2998: Add activity streams to group pages

seanh — Wed, 17 Oct 2012 14:33:14 GMT

Before we do this we should implement smarter group activity streams (i.e. what activities should appear in a group's activity stream?) and following of groups.

#2999: Add activity streams to organization pages

seanh — Wed, 17 Oct 2012 14:34:17 GMT

Before we do this we need to merge the organizations branch into master, we need to implement smarter activity streams for organizations, and we need to implement following of organizations.

#3000: Add number of followers to dataset pages

seanh — Wed, 17 Oct 2012 14:36:00 GMT

User profile pages now show some interesting stats about the user: num. datasets, num. edits and num. followers. Would be nice to add some similar info to dataset pages. At least num. followers can be done (already implemented in backend).

#3001: Multilingual plugin crashes CKAN on add dataset when some languages are default

seanh — Thu, 18 Oct 2012 16:50:48 GMT

Enable the multilingual plugins:

ckan.plugins = stats synchronous_search multilingual_dataset multilingual_group multilingual_tag

and set your default language to one not supported by the multilingual plugin, e.g.

ckan.locale_default = cs_CZ

now run CKAN and try to add a dataset:

File '/home/seanh/Projects/ckan171/ckan/ckanext/multilingual/plugin.py', line 141 in before_index

text_field_items+ default_lang?.extend(all_terms)

KeyError?: 'text_cs_CZ'

It doesn't matter what language you are viewing the site in in your browser, the default language setting in the ini file determines whether it crashed or not.

A number of supported languages are defined at the top of ckanext/multilingual/plugin.py. I think if the default language is not one of these it crashes.

I think this affects all versions of CKAN since the multilingual plugin was added so at least 1.7, 1.8 and 2.0

#3005: Add following/unfollowing of groups: model, API, frontend, tests

seanh — Tue, 23 Oct 2012 17:35:44 GMT

#3007: Adding a dataset creates multiple activities

seanh — Mon, 29 Oct 2012 13:15:07 GMT

Adding a new dataset creates multiple activity stream activities, e.g. seanh created the dataset foo, seanh add the resource bar to the dataset foo, and seanh updated the dataset foo are all created when I add a new dataset. I wonder if these can be collapsed into a single activity.

#3010: Pin images don't appear in data explorer

seanh — Wed, 31 Oct 2012 17:45:06 GMT

When minified files are in use (set debug = false in ini file) the pin images for points don't show up in the data explorer map view when viewing a resource with geolocation data.

set debug = true will fix the problem by using unminified files instead but also turns on a bunch of debug stuff you wouldn't want on a production site.

middleware.py, around line 73, is where the decision to turn minified files on or off based on the debug setting is made. This can be hacked on a production site to run with unminified files but otherwise run on production mode thereby getting around the issue with the pins. But that doesn't fix the underlying bug.

#3014: Crash when deleting a non-empty vocabulary

seanh — Tue, 13 Nov 2012 10:50:08 GMT

From Knud Möller:

when I try to delete a non-empty tag vocabulary via the API (through HTTP), I get an internal server error. Checking the logs, this turns out to be a consistency error raised by sqlalchemy:

Error - <class 'sqlalchemy.exc.IntegrityError?'>: (IntegrityError?) update or delete on table "vocabulary" violates foreign key constraint "tag_vocabulary_id_fkey" on table "tag" DETAIL: Key (id)=(21421955-7560-467c-af30-9f790b73e6ae) is still referenced from table "tag".

'DELETE FROM vocabulary WHERE vocabulary.id = %(id)s' {'id': u'21421955-7560-467c-af30-9f790b73e6ae'}

URL: http://33.33.33.10:5000/api/action/vocabulary_delete

The error makes sense, but I'm wondering if it would be useful to extend the API to also allow the deletion of non-empty vocabularies, possibly via a parameter (not sure what best practice in API design is). At the very least, it would be cool if the error message coming back in the response had more information in it.

#3019: Cannot delete dataset extras

seanh — Wed, 14 Nov 2012 18:44:38 GMT

Deleting extras in the web interface is broken

#3028: Feature: dashboard activity stream filtering

seanh — Mon, 10 Dec 2012 12:07:24 GMT

https://github.com/okfn/ckanext-pdeu/issues/13

#1785: Replace 'Revisions' page with site-wide activity stream

seanh — Tue, 07 Feb 2012 16:34:18 GMT

Replace the /revision page with a /activity page showing a site-wide activity stream of all activities in the site. Or perhaps the site-wide activity stream can go somewhere on the front page of the site instead?

#2243: Fix ckanext-example

seanh — Mon, 19 Mar 2012 15:58:30 GMT

#2335: Unicode characters don't work in CKAN ini files

seanh — Mon, 30 Apr 2012 12:42:06 GMT

You get a UnicodeDecodeError?, looks like it tries to decode the ini file as ascii. This is a problem because someone might try to set their site description to something with an umlaut in it, for example.

#2337: Sort out ini file confusion in docs

seanh — Mon, 30 Apr 2012 14:57:56 GMT

For example if you follow the source install instructions then you have development.ini, then when you go onto Post-Installation Setup it talks about std.ini. People don't realise that these files are the same.

Either fix the docs or fix CKAN to use same ini file names in package and source installs/