<?xml version="1.0"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" version="2.0">
  <channel>
    <title>CKAN: Ticket #1312: Particular characters in JSON cause exception creating package on API</title>
    <link>http://localhost/ticket/1312</link>
    <description>&lt;p&gt;
From Florian:
&lt;/p&gt;
&lt;pre class="wiki"&gt;I am having trouble with the characters "=&amp;amp;;".
if they occur e.g. in the title of a package, I get a 400-Bad-Request
when I try to register a package by POSTing to ckan/rest/api/package
However, when I PUT the same package as an edit to
ckan/rest/api/package/existing_package, it works and the title shows
correctly on the front end and API.
&lt;/pre&gt;&lt;h2 id="Reproduced"&gt;Reproduced&lt;/h2&gt;
&lt;pre class="wiki"&gt;(pyenv-ckan)dread@dread-laptop:~/hgroot/ckan$ curl -i http://localhost:5000/api/rest/package -d '{"name": "test3", "title": "Test &amp;amp;"}' -H "Authorization:tester"
HTTP/1.0 400 Bad Request
Server: PasteWSGIServer/0.5 Python/2.6.5
Date: Tue, 06 Sep 2011 14:09:31 GMT
Pragma: no-cache
Cache-Control: no-cache
Content-Type: application/json;charset=utf-8
Content-Length: 87
"Bad request - JSON Error: Unterminated string starting at: line 1 column 27 (char 27)"
&lt;/pre&gt;</description>
    <language>en-us</language>
    <image>
      <title>CKAN</title>
      <url>http://assets.okfn.org/p/ckan/img/ckan_logo_shortname.png</url>
      <link>http://localhost/ticket/1312</link>
    </image>
    <generator>Trac 0.12.3</generator>
    <item>
      
        <dc:creator>dread</dc:creator>

      <pubDate>Tue, 06 Sep 2011 16:49:03 GMT</pubDate>
      <title>status changed; resolution set</title>
      <link>http://localhost/ticket/1312#comment:1</link>
      <guid isPermaLink="false">http://localhost/ticket/1312#comment:1</guid>
      <description>
          &lt;ul&gt;
            &lt;li&gt;&lt;strong&gt;status&lt;/strong&gt;
                changed from &lt;em&gt;new&lt;/em&gt; to &lt;em&gt;closed&lt;/em&gt;
            &lt;/li&gt;
            &lt;li&gt;&lt;strong&gt;resolution&lt;/strong&gt;
                set to &lt;em&gt;invalid&lt;/em&gt;
            &lt;/li&gt;
          &lt;/ul&gt;
        &lt;p&gt;
The default Content-Type is "application/x-www-form-urlencoded" (that is what curl sends, and what CKAN defaults to). Therefore these characters need url encoding ("=&amp;amp;;" -&amp;gt; "%3D%26%3B"). Here is an example of it working:
&lt;/p&gt;
&lt;pre class="wiki"&gt;(pyenv-ckan)dread@dread-laptop:~/hgroot/ckan$ curl -i http://localhost:5000/api/rest/package -d '{"name": "test7", "title": "Test2 %3D%26%3B"}' -H "Authorization:tester"
HTTP/1.0 201 Created
Server: PasteWSGIServer/0.5 Python/2.6.5
Date: Tue, 06 Sep 2011 16:36:32 GMT
Pragma: no-cache
Cache-Control: no-cache
Location: http://localhost:5000/api/rest/package/5866a0f5-905a-44ff-91c3-c1489374a73e
Content-Type: application/json;charset=utf-8
Content-Length: 571
{"maintainer": null, "maintainer_email": null, "id": "5866a0f5-905a-44ff-91c3-c1489374a73e", "metadata_created": "2011-09-06T14:06:40.735488", "relationships": [], "metadata_modified": "2011-09-06T16:36:32.543461", "author": null, "author_email": null, "state": "active", "version": null, "license_id": null, "resources": [], "tags": [], "groups": [], "name": "test7", "license": null, "notes_rendered": "", "url": null, "notes": null, "title": "Test2 =&amp;amp;;", "ratings_average": null, "extras": {}, "ratings_count": 0, "revision_id": "92ee0b2e-c5e4-47b9-b00c-b8e29662b950"}
&lt;/pre&gt;&lt;p&gt;
Alternatively you can set the content type to "application/json" and you don't need to URL encode. e.g.:
&lt;/p&gt;
&lt;pre class="wiki"&gt;curl -i http://localhost:5000/api/rest/package -d '{"name": "test5", "title": "Test &amp;amp;"}' -H "Authorization:tester" -H "Content-Type: application/json;charset=utf-8"
&lt;/pre&gt;&lt;p&gt;
To help work out these problems with encoding in the future, I've improved the error message to show the decoded JSON string. cset:55273629a552 which is headed for CKAN 1.5 release.
&lt;/p&gt;
      </description>
      <category>Ticket</category>
    </item>
 </channel>
</rss>