{23} Trac comments (3729 matches)
Results (1201 - 1300 of 3729)
Ticket | Posixtime | Author | Newvalue | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
#1490 | 1322489155000000 | amercader | Fixed in c23821b | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1490 | 1324033557000000 | dread | Fixed in 1.5.1 * group_package_show introduced in 1.5.1 * tag_show added extended packages in 1.5 (e6bad88a) * package_search added extended packages in 1.5 (e6bad88a) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1489 | 1322138014000000 | dread | I updated the theme stuff in a75ca84. I've asked Ian to see if he can replace the form stuff with a new form (v. quick simple!). | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1488 | 1322241966000000 | dread | 'extensions' added. Also mounted RESTful version of status_show at: /api/util/status Done in cset:415d9ae aiming for 1.5.2 release. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1487 | 1322095808000000 | kindly | fixed cset:4160859c8c9786588dbf0893981b93d9621019a9 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1487 | 1324474147000000 | dread | Fixed in CKAN 1.5.1. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1486 | 1322061518000000 | seanh | Fixed here: https://github.com/seanh/ckan/tree/defect-1486 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1486 | 1322077524000000 | seanh | I've merged the fix for this into my feature-1298-activities-table branch, will merge it all into master at once: https://github.com/seanh/ckan/commits/feature-1298-activities-table | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1486 | 1323172027000000 | seanh | I've now merged my feature-1298-activities-table branch into my super branch for activity streams, so my fix for this defect is now merged into the activity streams super branch and will be merged when that branch is merged. See the activity streams super ticket: #1515 Resolving this ticket now | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1485 | 1323172388000000 | icmurray | Made a small start on this: https://github.com/okfn/ckan/tree/feature-1485-package-form-extension | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1485 | 1323186842000000 | ross | Implementing an initial IPackageController in ckanext-example | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1484 | 1322059006000000 | dread | Fixed and added tests. Improved IntegrityError error messages. Changeset:ddca1a7 for release 1.5.1. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1483 | 1323279151000000 | johnglover | Fixed. Updated logic layer: https://github.com/okfn/ckan/commit/520959ca6f83586d33cf7d4015d625a776ee46a1 Updated tasks (qa, archiver, webstorer): https://github.com/okfn/ckanext-qa/commit/78cbf5c6d16225e1580aee23fdf884f88d2475f5 https://github.com/okfn/ckanext-qa/commit/8553348942adad0babd4bdd91820c1d75a1784a7 https://github.com/okfn/ckanext-archiver/commit/8ed419280bd7f2e66aeb4f17f3f546d941fe11fb https://github.com/okfn/ckanext-webstorer/commit/d16fb090533a716c856e59838f79bbaa0ea42e76 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1481 | 1322062707000000 | johnglover | Fixed in new dataset UX changes (branch feature-1450-dataset-view), still has to be merged with master. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1480 | 1321978546000000 | dread | Done in /api/util. Added length limits to mungers. Also added markdown test and documentation. Cset:e43009db393cb for 1.5.2 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1479 | 1321961592000000 | dread | Fixed in cset:380eab620c7397 for CKAN 1.5.1. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1478 | 1323161054000000 | kindly | completed in about 2.8 days. cset:58b7a09328111b97da7d8ac65b4710b94dac54e3 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1477 | 1324549459000000 | dread | I added the sub-tickets since this tickets already covers them really. They may provide useful details in the course of doing all this. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1477 | 1325186158000000 | rgrp | @davidraznick: as per our discussion last week I think we need a ticket (or item) about writing up either in docs or in wiki how to implement various standard scenarios/workflows such as: * 'Curated tag' (standard) Groups * Publisher workflow - http://wiki.ckan.org/Workflows#Moderated_Workflow * DGU setup Suggest doing this either at end of http://wiki.ckan.org/Groups_Refactor, or perhaps better,http://wiki.ckan.org/Groups_Refactor. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1477 | 1328016209000000 | kindly | This is done pending new superticket publihser_profile. (#1669) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1475 | 1326283573000000 | ross | Initial draft written at http://wiki.ckan.org/Writing_asynchronous_tasks | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1474 | 1321826753000000 | kindly | fixed cset:8f3d917e24390f91db577fdbd8b8c6a1d6228505 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1473 | 1321872717000000 | dread | Done in cset:7733eb19971ff06e for 1.5.2 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1472 | 1321721874000000 | rgrp | Done yesterday. See https://github.com/okfn/ckanclient | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1471 | 1340623578000000 | seanh | Also test the packaging tools | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1471 | 1340633734000000 | seanh | The source install instructions have been updated to work with both lucid and precise, see #2592 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1470 | 1321540943000000 | dread | Adria put in fix cset:ad29e3ea75 but the model change missed a migration script and left some spurious tests broken. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1470 | 1321543837000000 | dread | Tidy up done in cset:c46388b945d581 for 1.5.1 release and merged to master. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1470 | 1324473955000000 | dread | This issue only affects 1.5 release. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1469 | 1329760150000000 | amercader | This is mostly done (current form is http://i.imgur.com/zmfc5.png). Still some tests missing and a little bit of cleanup and documentation required. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1468 | 1321875777000000 | johnglover | New ticket, moving over to new sprint | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1468 | 1322495417000000 | johnglover | Done - commit: https://github.com/okfn/ckan/commit/7789e85c973c9e085f623486bced6be14f25678f rebuild can now take an optional package name/id (single package to be consistent with other paster commands, not a list of packages) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1468 | 1322591997000000 | dread | We originally talked about a command-line interface for deleting packages. I've done this here: #1499. Note: you can update the search index from a paster shell, simply by doing this before running your commands that edit packages: {{{ from ckan import plugins plugins.load('synchronous_search') }}} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1467 | 1321872507000000 | dread | Leaving this to James to schedule | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1467 | 1325462196000000 | rgrp | Moving to current sprint as this sprint is now long finished. @jimmyg: please close, defer, update as necessary! | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1467 | 1326104869000000 | thejimmyg | The publisher issue seems to be resolved now, although during investigation I also found these issues: * 9 of the records don't have a published by and I wondered why * Lots of them are state=deleted (so do we really want to include these?) * We're still showing the deprecated agency field * Many of the departments are blank Pawel is not available to work on these anyway at the moment, so let's pick them up as part of the disintegration work to migrate to CKAN. Marking the main ticket as "worksforme" since it does now. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1467 | 1326119954000000 | dread | I believe that the "publisher issue" that James alludes to is that the dump doesn't contain the 'parent publisher' field that is generated in the DGU system on the Drupal side. This information will be stored following the Groups Refactor #1477 and should be added to the dump at this point. Excluding Datasets that are state=deleted is a good idea. I've split that off into #1623 The other issues mentioned are simply data quality - the same whether viewing the dump or elsewhere. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1467 | 1326120319000000 | dread | > 'parent publisher' Sorry, I meant 'parent department' | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1466 | 1323710030000000 | dread | Is this still critical, James? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1466 | 1324298713000000 | rgrp | Moving to v1.6 as no specific milestone for this. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1466 | 1328529062000000 | rgrp | Moving to backlog. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1465 | 1323710679000000 | dread | Pls update status and milestone. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1464 | 1323169763000000 | thejimmyg | As part of the queue upgrade we'll also fix #1064 which explains that the current queue implementation is over-engineered. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1464 | 1323710659000000 | dread | Pls update status and milestone | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1464 | 1328529042000000 | rgrp | Closing as wontfix as no further info and seems unimportant. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1463 | 1321874259000000 | johnglover | This works with the new Celery feature that will be in 1.5.1 (which should be released in this sprint). So, will not update this old version of QA for 1.5, people should use the new version (on okfn Github) after 1.5.1 is released. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1462 | 1323172211000000 | thejimmyg | Let's work together to fix the packaging aspect too. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1462 | 1326283656000000 | rgrp | @amercader: update? Could you also either close or move sprint (or to backlog as appropriate). | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1462 | 1326284357000000 | amercader | Closing as this has been fixed and deployed. @thejimmyg Not sure if there are still issues regarding packaging. Feel free to create a specific ticket for this if we need to work on it. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1462 | 1330083671000000 | dread | This went into CKAN 1.5.1. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1461 | 1321359503000000 | dread | Fixed in cset:6ea5d3c50444 so all functions supply api key. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1460 | 1328001114000000 | rgrp | @dread: re-assigning to you (at least to review). Would be really good to have this closed out asap. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1460 | 1328638393000000 | dread | Have not looked at this yet. Bumping to this sprint. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1458 | 1340632821000000 | icmurray | Re-assigned to amercader. Kept on ckan-future. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1458 | 1340632932000000 | icmurray | Sorry, I shouldn't have touched this. I pulled it from the wrong milestone. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1456 | 1320930770000000 | dread | I see you've done this in cset:939e0e0809c1. Close now? BTW Take a look at the first suggestion from #1423 too, whilst in the area. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1456 | 1324472178000000 | dread | Went into 1.5.1 release | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1455 | 1321872633000000 | dread | John to look at. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1455 | 1322491506000000 | johnglover | Fixed - commit: https://github.com/okfn/ckan/commit/05b675a4314ad269c6e6a095d57e3f2a21e771eb Note: - Includes a small change to Solr schema file. - Search index will need to be rebuilt for changes to take effect | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1455 | 1324474466000000 | dread | Fix has gone into CKAN 1.5.1 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1454 | 1320849527000000 | dread | Done in cset:138c5daf7765 heading for release 1.5.1. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1453 | 1320947955000000 | icmurray | Added the restriction of not allowing the double quote character, '"', as well as commas as this simplifies any use of quoting multiple words to mean a single tag name. For example, this simplifies the use of quotes in identifying tags in internal markdown links: {{{ tag:"multiple word tag name" }}} A possible solution is to allowing escaping, such as: {{{ tag:"something about \"Ian\"" }}} But I think the compromise is a better solution than allowing the escaping as it's simpler, and this may crop up elsewhere. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1453 | 1321548452000000 | icmurray | The allowable characters in a tagname has changed to "unicode alphanmeric plus simple punctuation". This means: - alphanumeric (inc. foreign characters) - [ .-_] The completed feature is in the [https://github.com/okfn/ckan/tree/feature-1453-flexible-tag-names feature-1453-flexible-tag-names branch]. Awaiting a code review. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1453 | 1321635178000000 | dread | Code review: * basically - really excellent code and very thorough :-) * links should have %20 rather than spaces (tests/misc/test_format_text.py:61) * also check unicode chars encoding in urls (tests/misc/test_format_text.py:115) * also check searching for the tag with this encoding (ckan/tests/functional/api/model/test_tag.py:35) * we follow the PEP8 coding style which I interpret to mean not having blank lines after a function definition. But whichever, we're not consistent from file to file, but we should be within each file. e.g. ckan/tests/forms/test_package.py:12. * moo package problem - need to ensure test works on its own and when run as part of the suite, so independent of whether moo exists. tests/functional/api/test_action.py: * best to make tag search case insensitive - see ckan/tests/functional/api/model * It's worth keeping the old test in addition to your modified one - because query for just {q:''} will return both packages too. ckan/tests/functional/api/test_package_search.py:203 * Let's add an example of tag search with quotes in /doc/api.rst:337 * Please put imports at the top of the file, unless there's a good reason ckan/tests/functional/api/test_package_search.py:296 * Can you not the old test any more? It seems sufficiently to the test you changed it to, so can we include both? ckan/tests/functional/api/test_package_search.py:295 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1453 | 1321965613000000 | icmurray | Updated code now in feature-1453-flexible-tag-names branch. (Also, deleted the ian-review branch.) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1453 | 1322581830000000 | dread | I believe this is finished now. This was merged into master in cset:c0aaa31c4b7ded54d and headed for release 1.5.2. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1453 | 1329395697000000 | dread | This has gone into release 1.6. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1452 | 1321276189000000 | dread | Seb Bacon: > I agree. It is quite standard for people to have their browser > language as en-US in many countries. > Yes, geo-location is likely to be better if you want to automate it. > > http://www.maxmind.com/app/geoip_country > > But I wouldn't automatically detect it at all. Just have a default > language for each site, as you suggest. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1452 | 1340190587000000 | ross | I believe this has been addressed by Toby. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1451 | 1321876014000000 | johnglover | Haven't looked at this yet, moving to new sprint | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1451 | 1322567112000000 | johnglover | This was taken from the UX pad. I'm looking at it now to make sure that it's working and that we can use it on the new dataset view page. Is it really a good idea to move it to core though? There was no reason given on the pad so I'm not sure why this should be moved to core? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1451 | 1322568784000000 | dread | With the cron job setup, maybe this is best left as an extension. The core CKAN was designed to be mean and lean. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1451 | 1323165781000000 | johnglover | Scripts and templates updated for CKAN 1.5.1 Waiting for Tom to finish dataset view and resource view updates, then will discuss best place to put the download stats on the page. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1451 | 1324288407000000 | johnglover | Need to add analytics javascript to new resource view page, will do under ticket #1519. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1451 | 1324401792000000 | johnglover | This ticket again brought up the issue of a need for inversion of control for writing to CKAN templates (main templates should really ask extensions for data or provide hooks rather than having extensions overwrite sections using genshi/jquery). Have decided to defer this issue for now, as it will be looked at when looking at making our whole extension system more robust early next year. Related point: * We should provide a dashboard area (again via interfaces/hooks) that groups together report data from related extensions. So stats and googleanalytics info should be available from a similar area / url. This is also deferred until the extension overhaul/refactor. Other outstanding issues: * Location / style of download count can be improved * Download count should be shown on resource pages These should be looked at by whoever takes over frontend design. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1450 | 1321875986000000 | johnglover | Still working on this, moving to new sprint | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1450 | 1323109665000000 | zephod | johnglover architected this and I've added aesthetic tweaks. Now pushed on branch: feature-1450-dataset-view. One bug ticket remains (#1517) before the work can be merged. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1450 | 1330019974000000 | dread | In CKAN 1.6. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1449 | 1321465348000000 | johnglover | Basic change done in branch feature-1449-resource-listing. Currently showing: * resource name (clickable) or (none) * resource description * resource format * resource last modified (if exists) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1448 | 1325774155000000 | rgrp | @kindly: Any details ;-) (even when this happened, approach taken)? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1322220690000000 | dread | This appears to have happened again today on test.ckan.net and someone has sorted it. The problem is visible on munin as inodes running out. * eu25 seems ready to fall over in about a week: http://munin.okfn.org/okfn.org/eu25.okfn.org-df_inode.html * thedatahub.org on s055 (and other fry instances) seem to have dynamically adjusted inode table size (by the kernel) so is less of a problem | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1322648873000000 | dread | As predicted, this happened again today. From the following analysis it confirms that the problem is the cache growing and growing. Disk usage in megabytes: {{{ okfn@s025:~/var/srvc/publicdata.eu$ du -s -m /* 7 /bin 22 /boot 1 /dev 10 /etc 4157 /home 0 /initrd.img 0 /initrd.img.old 114 /lib 1 /lost+found 1 /media 1 /mnt 1 /opt 0 /proc 1 /root 7 /sbin 1 /selinux 1 /srv 0 /sys 1 /tmp 421 /usr 443 /var 0 /vmlinuz 0 /vmlinuz.old }}} {{{ okfn@s025:~/var/srvc/publicdata.eu$ du -s -m /home/okfn/var/srvc/publicdata.eu/*2173 /home/okfn/var/srvc/publicdata.eu/backup 1 /home/okfn/var/srvc/publicdata.eu/backup_RENAMED_TO_AVOID_MAYHEM.sh 1 /home/okfn/var/srvc/publicdata.eu/common.sh 1893 /home/okfn/var/srvc/publicdata.eu/data 1 /home/okfn/var/srvc/publicdata.eu/fetch.sh 1 /home/okfn/var/srvc/publicdata.eu/gather.sh 1 /home/okfn/var/srvc/publicdata.eu/pip-requirements.txt 1 /home/okfn/var/srvc/publicdata.eu/publicdata.eu.ini 86 /home/okfn/var/srvc/publicdata.eu/pyenv 1 /home/okfn/var/srvc/publicdata.eu/run.sh 1 /home/okfn/var/srvc/publicdata.eu/sstore 0 /home/okfn/var/srvc/publicdata.eu/who.ini }}} {{{ okfn@s025:~/var/srvc/publicdata.eu$ ls -l /home/okfn/var/srvc/publicdata.eu/backup total 2224588 -rw-r--r-- 1 okfn okfn 343199744 2011-06-14 20:50 db-20110614-2050.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 20:51 db-20110614-2051.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 20:52 db-20110614-2052.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 20:53 db-20110614-2053.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 20:54 db-20110614-2054.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 20:55 db-20110614-2055.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 20:56 db-20110614-2056.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 20:57 db-20110614-2057.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 20:58 db-20110614-2058.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 20:59 db-20110614-2059.sql -rw-r--r-- 1 okfn okfn 1036288 2011-06-14 22:00 db-20110614-2200.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:01 db-20110614-2201.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:02 db-20110614-2202.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:03 db-20110614-2203.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:04 db-20110614-2204.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:05 db-20110614-2205.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:06 db-20110614-2206.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:07 db-20110614-2207.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:08 db-20110614-2208.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:09 db-20110614-2209.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:10 db-20110614-2210.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:11 db-20110614-2211.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:12 db-20110614-2212.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:13 db-20110614-2213.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:14 db-20110614-2214.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:15 db-20110614-2215.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:16 db-20110614-2216.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:17 db-20110614-2217.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:18 db-20110614-2218.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:19 db-20110614-2219.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:20 db-20110614-2220.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:21 db-20110614-2221.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:22 db-20110614-2222.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:23 db-20110614-2223.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:24 db-20110614-2224.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:25 db-20110614-2225.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:26 db-20110614-2226.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:27 db-20110614-2227.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:28 db-20110614-2228.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:29 db-20110614-2229.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:30 db-20110614-2230.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:31 db-20110614-2231.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:32 db-20110614-2232.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:33 db-20110614-2233.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:34 db-20110614-2234.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:35 db-20110614-2235.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:36 db-20110614-2236.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:37 db-20110614-2237.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:38 db-20110614-2238.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:39 db-20110614-2239.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:40 db-20110614-2240.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:41 db-20110614-2241.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:42 db-20110614-2242.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:43 db-20110614-2243.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:44 db-20110614-2244.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:45 db-20110614-2245.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:46 db-20110614-2246.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:47 db-20110614-2247.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:48 db-20110614-2248.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:49 db-20110614-2249.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:50 db-20110614-2250.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:51 db-20110614-2251.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:52 db-20110614-2252.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:53 db-20110614-2253.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:54 db-20110614-2254.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:55 db-20110614-2255.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:56 db-20110614-2256.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:57 db-20110614-2257.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:58 db-20110614-2258.sql -rw-r--r-- 1 okfn okfn 0 2011-06-14 22:59 db-20110614-2259.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:00 db-20110615-0000.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:01 db-20110615-0001.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:02 db-20110615-0002.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:03 db-20110615-0003.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:04 db-20110615-0004.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:05 db-20110615-0005.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:06 db-20110615-0006.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:07 db-20110615-0007.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:08 db-20110615-0008.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:09 db-20110615-0009.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:10 db-20110615-0010.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:11 db-20110615-0011.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:12 db-20110615-0012.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:13 db-20110615-0013.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:14 db-20110615-0014.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:15 db-20110615-0015.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:16 db-20110615-0016.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:17 db-20110615-0017.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:18 db-20110615-0018.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:19 db-20110615-0019.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:20 db-20110615-0020.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:21 db-20110615-0021.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:22 db-20110615-0022.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:23 db-20110615-0023.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:24 db-20110615-0024.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:25 db-20110615-0025.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:26 db-20110615-0026.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:27 db-20110615-0027.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:28 db-20110615-0028.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:29 db-20110615-0029.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:30 db-20110615-0030.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:31 db-20110615-0031.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:32 db-20110615-0032.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:33 db-20110615-0033.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:34 db-20110615-0034.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:35 db-20110615-0035.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:36 db-20110615-0036.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:37 db-20110615-0037.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:38 db-20110615-0038.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:39 db-20110615-0039.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:40 db-20110615-0040.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:41 db-20110615-0041.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:42 db-20110615-0042.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:43 db-20110615-0043.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:44 db-20110615-0044.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:45 db-20110615-0045.sql -rw-r--r-- 1 okfn okfn 0 2011-06-15 00:46 db-20110615-0046.sql -rw-r--r-- 1 okfn okfn 483144447 2011-06-15 10:00 db-20110615-1000.sql -rw-r--r-- 1 okfn okfn 482136064 2011-06-15 10:07 db-20110615-1007.sql -rw-r--r-- 1 okfn okfn 483144447 2011-06-15 10:50 db-20110615-1050.sql -rw-r--r-- 1 okfn okfn 483053568 2011-06-15 10:51 db-20110615-1051.sql }}} {{{ okfn@s025:~/var/srvc/publicdata.eu$ du -s -m /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/* 117 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/0 116 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/1 117 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/2 116 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/3 116 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/4 116 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/5 116 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/6 117 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/7 116 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/8 117 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/9 117 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/a 116 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/b 116 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/c 116 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/d 117 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/e 116 /home/okfn/var/srvc/publicdata.eu/data/sessions/container_file/f }}} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1324374324000000 | dread | Once again eu25 ran out of space again today. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1326279495000000 | dread | eu25 ran out of space again this weekend and eu8 (at/it/us_co) today. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1326296987000000 | nils.toedtmann | For time being, i created a cron script [https://bitbucket.org/okfn/sysadmin/src/default/etc/cron/remove_old_files remove_old_files]. You could just copy it to /etc/cron.daily/, but i recommend to not run it as root: if it's misconfigured, it could wipe the system! So you better copy it to /home/okfn/sbin/ (not /home/okfn/bin/ which often is the sysadmin HG repo), and add it to some unprivileged user's crontab. In most cases, the leftover files are owned by user "www-data", so {{{ $ sudo crontab -e -u www-data }}} and then add something like {{{ 37 4 * * * /home/okfn/sbin/remove_old_files }}} Don't forget to edit the script remove_old_files itself and list the directories you want to be cleaned up. This is already done on s008/eu8 and s019/eu19. dread, do you want to do this for s025/eu25 and see how this goes? ---- Todo nils: verify tomorrow on s019 that it worked properly, e.g. this should show only a few files: {{{ find /var/lib/ckan/nederland/data/sessions/ -type f -amin +$((7*24*60)) -ls }}} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1330082636000000 | nils.toedtmann | I had forgotten to check s019 how well my cleanup script is working (and now s019 is gone), but at least it didn't destroy it :-) You might want to give it a try on s025/PDEU. (Tell me if you want me to do that). | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1330082808000000 | dread | Yes please Nils! | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1330089662000000 | nils.toedtmann | OK i fixed a bug in my script and refactored it so that it can now be dropped into /etc/cron.daily/ while still deleting as unprivileged user. It is now running on s025, removing everything older than 7 days. Please verify in 9 days or so that it's working. Consider to add [https://bitbucket.org/okfn/sysadmin/src/default/etc/cron/remove_old_files this cron job] to the ckan deb package e.g. as "/etc/cron.daily/ckan-cleanup" | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1332510790000000 | nils.toedtmann | Just checked s025 (which is depricated now), looks like my script is working fine - nothing older than a week in /home/okfn/var/srvc/publicdata.eu/data/sessions/. We should activate this script on other hosts as well, e.g. so55/thedatahub. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1332510913000000 | nils.toedtmann | Just to add: the remove_old_files script is only a workaround, not a fix. CKAN should clean up after itself. Feel free to re-open this ticket for a proper solution ;-) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1332511544000000 | rgrp | @kindly: hope ok to assign to you (maybe just for review and thought on who would be best placed to look at ...) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1332519029000000 | nils.toedtmann | Ticket http://trac.okfn.org/ticket/1222 tracks the effort to push the clean-up script onto CKAN hosts. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1340726283000000 | nils.toedtmann | This is becoming painful for the sysadmins. Please fix. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1447 | 1340727330000000 | dread | BTW on DGU I have set it up to use memcached for these sessions (v. easy) and I think it solves the problem. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1446 | 1321872724000000 | rgrp | Have made significant progress on new recline but not yet operational. See https://github.com/okfn/recline and https://github.com/okfn/recline/issues/6 https://github.com/okfn/recline/issues/12 https://github.com/okfn/recline/issues/8 also https://github.com/okfn/recline/issues/10 and more ... | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1446 | 1326281594000000 | rgrp | Marking as complete as Data Explorer is now functional enough for our purposes. Tickets done include (* indicates improvement over current explorer) * Core Backbone Models representing Dataset and Tabular data: https://github.com/okfn/recline/issues/10 * New theme: https://github.com/okfn/recline/issues/22 * Read-only mode: https://github.com/okfn/recline/issues/17 * Introduce hash navigation / state support (*): https://github.com/okfn/recline/issues/19 * Re-enable editing in DataTable (*): https://github.com/okfn/recline/issues/13 * [super] DataTable view (in Backbone) (*): https://github.com/okfn/recline/issues/14 * DataExplorer parent view (*): https://github.com/okfn/recline/issues/12 * Simple graph widget using flot: https://github.com/okfn/recline/issues/11 NB: quite a bit of work done pre 2012-01-09 iteration but in that iteration that we finished the work. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1446 | 1326281658000000 | rgrp | Total time: * 2012-01-09: 4.5d * Pre that date: 5d | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1445 | 1323178176000000 | zephod | JohnGlover has done most of this and I've been adding aesthetic tweaks. The resource page is now tidy and has a similar look and feel to its parent Dataset view page. I'm closing this ticket and its sibling #1450; I am merging the feature-1450-dataset-view branch which contains all of this work. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1445 | 1330019916000000 | dread | This went into CKAN 1.6 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
#1444 | 1323089960000000 | dread | I've not looked at this yet, but now detection is disabled, less important. |
Note:
See TracReports for help on using and creating reports.