{22} Trac tickets (2647 matches)

With an IDatasetForm plugin active with a form_to_db_package_schema() method that returns a schema based on ckan/logic/schema.py:form_to_db_package_schema() I'm getting validation errors like this:

[ckan.controllers.api] Validation error: "{'type': 'Validation Error', 'junk': [u'The input field junk was not expected.']}"

They seem to be caused when a data dict with key 'tags' is passed to e.g. the package_update API, for example in ckan/tests/functional/api/model/test_vocabulary.py there are various tests that get this error when they call package_update (if you run the tests with a suitable IDatasetForm plugin active).

In ckan/logic/schema.py:form_to_db_package_schema() 'tags' is for some reason deliberately removed from the schema.

And as a result timeout on e.g. the datahub ...

If there's an IDatasetForm plugin active with a db_to_form_schema() method based on db_to_form_package_schema(), then viewing historical versions of datasets (e.g. ​http://127.0.0.1:5000/dataset/dasd%402012-08-30T12%3A26%3A18.524191) crashes.

Reason is that revison_id and revision_timestamp need to be added to db_to_form_package_schema(), something like:

schema.update({'revision_id': [validators.ignore_missing, unicode]}) schema.update({'revision_timestamp':

[validators.ignore_missing, unicode]})

Goto:

1. Add dataset

2. Get to step 2

3. Click "Save & add another" without adding any data

4. Ends on step 3

Should: fail to validate and remain on step 2

The language dropdown in Internet Explorer 7 behaves oddly. Essentially I think the problem is with the way that the dropdown decides to go into "drop-up mode" (e.g there aint enough screen space below the initial state so it has to drop up).

On the search results page when there are no more formats, groups or tags to display it should not display the show more links.

The flags in the dropdown on details pages are not right.

You can see the error on ​http://s031.okserver.org:2375/dataset/malawi-aid-projects/resource/b717c20e-2006-4ad4-82d2-59b57ebc1ab0.

They seem to be not exactly well thought out. Needs some thought on better display of the information that they contain.

A summary/user story doc of how organizations and groups are expected to work.

Should be group_form() I think

Having two methods form_to_db_schema(self) and form_to_db_schema_options(self, options) seems unnecessary, why not just have form_to_db_schema(self, options=None)?

Fixing this might break existing extensions although I don't think any are using form_to_db_schema_options() so it should be okay.

The same thing goes for db_to_form_schema_options().

Also why are we passing a dict 'options' that always contains the same three keys 'api', 'context' and 'type'? Why not three params api=None, contenxt=None, type=None?

In setup.py, do we still need:

('templates/.txt', 'genshi', {

'template_class': 'genshi.template:TextTemplate'

}),

in message_extractors? Doesn't like there are any txt files in there anymore.

I'm unsure about what's going on here. As I understand it, when we run python setup.py extract_messages it's going to use the extract_ckan() function from ckan/lib/extract.py to process the HTML files. For the HTML files that are jinja2 templates, extract_ckan() will call jinja2_cleaner() which will call regularise_html() on the strings. So the strings are regularised when they are extracted from the source files.

But then we have the parse() method of CkanInternationalizationExtension? also calling regularise_html(). I don't get what's happening here. Why do the strings need to be regularised twice?

My guess is that CkanInternationalizationExtension? is used when the strings are extracted from the templates at runtime, and they need to be regularised at this time in order to match them against the regularised strings in the mo files to find the translations to output?

Maybe CkanInternationalizationExtension? needs a better docstring saying what it does?

Add a better docstring to for trans js command explaining what it does and why, and how to use it.

Also add this command to the CKAN Release Process as it's needed there

Finish the tests for the per-package-type custom template support in IPackageForm, IGroupForm and IOrganizationForm (i.e. the new_template(), read_template() etc. methods).

The new way that template inheritance works in jinja looks like it should allow this to be tested nicely without duplicating entire templates in the example extensions.

Before releasing CKAN 2.0 we need to run all the CKAN tests with a modified test-core.ini with the example_idatasetform, example_igroupform and example_iorganizationform plugins enabled. If any tests fail, fix the bugs. This needs to be done for each release so add it to the release process.

Use an IGroupForm plugin to add a custom metadata field to groups using convert_to_extras() and convert_from_extras(), when calling group show the value comes back quoted, e.g. '"my_value"'

Should add tests to example_igroupform and others that setting and getting the custom fields works through the action API.

What is the system_info table for? It looks like a way to override config file settings in the database, perhaps so that admins can use a web interface to change site title etc. Could do with a docstring explaining the intended purpose of the table.

The tests in CKAN only test single pages (functional tests) or single methods (unit tests). In order to make sure that the whole system still works as expected, we need tests that cover whole processes.

An example user story to be tested would be: "As a user with create permissions I want to be able to log in, create a new resource, save it and then make sure that it is in the system." or: "As a user I want to use the search functionality and open a resource from the search page."

These user stories require a different testing system with interaction and processes.

GroupController?'s history() method doesn't pass 'extras_as_string': True in the context when it calls group_show. This means that if you have an IGroupForm plugin that is adding a custom metadata field and using convert_to/from_extras() then a field value of 'foo' will be returned as '"foo"'

Other GroupController? and PackageController? methods do pass 'extras_as_string'

Opera cannot show a recline preview. I shows 0 records.

The pdf preview does not load in IE 9.

Recline allows negative ranges. The backend returns an error 500 for that which breaks the recline preview.

With the latest template, css and js changes in 2.0, there are a number of things that need preparation prior to a production deployment. One of these is:

• the less files should be compiled to css (main.css, not just main.debug.css)

This ticket is to provide a paster command which will compile the .less into the main.css file. The idea of the paster command is that it will be run by developers, and they will check in the resulting .css files. It will also provide a convenient mechanism for production scenarios with node.js installed on the production server, as they will be able to compile the .less there as well.

## Background

The .css files that need to be served are built using less. The css files that are generated *are* checked-in to the repo; but they are a build artifact. So the general workflow is:

• commit changes to the less files
• build the main.css file and commit

This paster command slots into the above workflow for convenience.

## Notes

• doc/frontend-development.rst
• bin/less

to run this paster command, Node will be required. So that dependency should be checked.

## The paster command

This is the proposed behaviour of the paster command:

• ensure custom.less does not exist
• for each colour in{fuchsia,green,maroon,red}:
  • generate a custom.less file for $colour. There's a paster command that does this already: `paster color <color name>.
  • generate the css from the less files. ie - the equivelant of running bin/less --production.
    • this will generate a main.css file, which should be renamed to $color.css.

• ensure custom.less does not exist
• generate the css from the less files. This will create a main.css with the default colour scheme.

• call the paster command that minifies css and js files. (This command does not exist yet, see #2950)

The GRANT SELECT ON ALL and ALTER DEFAULT PRIVILEGES are not available in Postgres prior to version 9.0.

At the moment everyone can read all resources. There is no support for hiding resources that are in private datasets.

e.g. ​http://thedatahub.org/tag/lod

Tags with less datasets work fine (e.g. ​http://thedatahub.org/tag/railways)

• Graphs not showing (looks like a flot related file is missing)
• Wrong groups counts (e.g. Data Explorer Examples show 1800 datasets when it onlu has 8)

Edit an organizations members page and demote yourself from organization admin, after saving the members edit page reloads even though you are no longer an admin and should no longer be able to access this page.

The group members page (e.g. /group/members/roger) seems broken, IDs instead of user names are shown for the members, and clicking on a member 500s

when deleting members from groups and orgs

Makes sense to keep all auth stuff under logic/auth.

Also decide which functions from new_authz.pu should be marked private with _.

Also decide which functions from new_authz.py are really helpers for the auth functions in get.py, update.py and delete.py, and should maybe moved to another file, e.g. logic/auth/helpers.py

From Toby: tests around the auth functions and new actions - probably needs an improved testing mechanism and things like test data creation on via actions

From Toby: @johnmartin the user autocomplete on member add needs fixing it gets data but does not understand what to do with it

From Toby: The requesting membership user stories and implementation remain needed

From Toby: remove the config from jinja2 templates get all such stuff via g. (before 2.0)

From Toby: c) refactor lib/base.py to remove the circular import issues (render functions to lib/render.py - would fix much of that)

From Toby: make lib/helpers more template specific and less generally shared with none template uses (some functions are shared but many shouldn't be) - again help avoid circular imports

From Toby

'Add dataset to organization' button, when you get through to the third stage of the new dataset form the organization you came from is not selected.

I don't see any option to choose the group when adding or updating a dataset, but if I add a dataset via the "Add dataset to group" button on a group's page, then the dataset seems to get added to that group. (And I can also add/remove existing datasets by editing the group.) I wonder if organizations should work the same way, instead of having an Organization drop-down when creating or updating a dataset. The add dataset page needs to somehow indicate that you're adding a dataset to a certain group or organization though, doesn't currently.

Currently if you want to customise the header.html template you have to copy the whole thing, it isn't broken up into blocks like other templates are.

If you hover the mouse over a dataset or user in an activity stream, you now get a popover with some info about that dataset or user and a follow/unfollow button.

It would be nice to add similar popovers for other types of object in CKAN, e.g. resources, groups, organizations, tags...

Add an activity stream tab to dataset pages, like we have on user profile pages. Dataset activity streams are already implemented in the backend.

Before we do this we need to merge the organizations branch into master, we need to implement smarter activity streams for organizations, and we need to implement following of organizations.

Enable the multilingual plugins:

ckan.plugins = stats synchronous_search multilingual_dataset multilingual_group multilingual_tag

and set your default language to one not supported by the multilingual plugin, e.g.

ckan.locale_default = cs_CZ

now run CKAN and try to add a dataset:

File '/home/seanh/Projects/ckan171/ckan/ckanext/multilingual/plugin.py', line 141 in before_index

text_field_items+ default_lang?.extend(all_terms)

KeyError?: 'text_cs_CZ'

It doesn't matter what language you are viewing the site in in your browser, the default language setting in the ini file determines whether it crashed or not.

A number of supported languages are defined at the top of ckanext/multilingual/plugin.py. I think if the default language is not one of these it crashes.

I think this affects all versions of CKAN since the multilingual plugin was added so at least 1.7, 1.8 and 2.0

Just to track @tauberer patch on Github. Probably going to 1.8.1

There are some new translations already on the CKAN 1.8 resource on Transifex already (e.g. 100% Norwegian translation I think). We should at least pull these from Transifex for 1.8.1, and may as well do a call for translations too.

We need to test that only logged in users can see their own dashboard activity streams, and they can only see their own dashboard activity streams not those of other users.

From Knud Möller:

when I try to delete a non-empty tag vocabulary via the API (through HTTP), I get an internal server error. Checking the logs, this turns out to be a consistency error raised by sqlalchemy:

Error - <class 'sqlalchemy.exc.IntegrityError?'>: (IntegrityError?) update or delete on table "vocabulary" violates foreign key constraint "tag_vocabulary_id_fkey" on table "tag" DETAIL: Key (id)=(21421955-7560-467c-af30-9f790b73e6ae) is still referenced from table "tag".

'DELETE FROM vocabulary WHERE vocabulary.id = %(id)s' {'id': u'21421955-7560-467c-af30-9f790b73e6ae'}

URL: ​http://33.33.33.10:5000/api/action/vocabulary_delete

The error makes sense, but I'm wondering if it would be useful to extend the API to also allow the deletion of non-empty vocabularies, possibly via a parameter (not sure what best practice in API design is). At the very least, it would be cool if the error message coming back in the response had more information in it.