Looking for help with duplicates

  • I made a mistake with conquest.


    I ran conquest for a long time as .dcm jpg lossless, I ran into problems with out of memory errors and tried running as .v2 nki and .dcm uncompressed.


    To make a long story short, I managed to drag and drop incorrectly and now have a somewhat mixed batch of files that include many (30% or more) duplicates. That is, any given patient directory may have duplicated .v2 and .dcm files.
    To make it worse, a duplicated large MRI or CT sequence locks up conquest/mysql (even with the rapid disconnect fixes put in). So trying to drag and drop into conquest is difficult as it locks every few patients and requires system reboot.


    There are over a million files, too many to do by hand. My backup mirror is equally mucked up


    Does anyone know of any dicom tools or conquest tips that may be helpful in condensing and cleaning up my image set?


    Thanks, LJJ

  • Hi,


    My advice is to go and buy an extra big harddrive (maybe 2, one for a full backup before changing anything) and move the data to a new MAG device. This will leave the duplicates behind. Or transmit the data patient by patient to a new server (dbase III database) to clean it up.


    Marcel

  • Marcel,
    The reason that the duplicates were a problem was that dragging/dropping resulted in conquest freezing after some amount of files. The problem seemed to happen most where I had a large number (CT or MRI) of duplicated files. So...
    I set up a 14b conquest DBIII and drag/dropped the directory into it as suggested in your post. The behavior I see is the same as with MySQL. That is, after some amount of time, conquest freezes. If I close dgate, dgateserv, conquestDICOMserver, it is no avail. Any attempts to drag/drop immediately lock up conquest and I have to reboot to regain functionality. I think the problem is due to malformed DICOM files (GE US?), but this is very hard to troubleshoot as I don't get any log files after the freeze.
    I thought this was a MySQL problem, but the system behaves the same with the DBIII driver. and shutting down and restarting dgate does not help...
    Any ideas on where this problem is?


    Any one else struggle with a similar issue?


    LJJ

  • Hi,


    Sound like an OS issue. You can create a huge batch file to load each file with dgate --loadimagefile:. If this works, it is purely a drag and drop problem. We use loadimagefile to load each file when dragging and dropping.


    Marcel

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!