Recover Lost Files And Folders
Perhaps the most common type of filesystem problem is files that are accidentally deleted. Users frequently delete the wrong files or delete a file only to discover that it's actually needed. Windows system users may be accustomed to undelete utilities, which scour the disk for recently deleted files in order to recover them. Unfortunately, such tools are rare on Linux. You can make undeletion easier by encouraging the use of special utilities that don't really delete files, but instead place them in temporary holding areas for deletion later. If all else fails, you may need to recover files from a backup.
His chapter examines the practice of safeguarding data through backups, restoring that same data if necessary, and recovering data in case of a catastrophic hardware or software failure. After reading this chapter, you will have a full understanding of the reasons for sound backup practices. You can use the information in this chapter to make intelligent choices about which strategies are best for you. The chapter also shows you how to perform some types of data recovery and system restoration on your own and when to seek professional assistance.
Files, as well as extract individual files from them. As if that was not enough, mc contains a File Undelete virtual file system for ext2 3 partitions. By using cd to change directories to an FTP server's URL, you can transfer files using the FTP protocol. The default font chosen for Ubuntu makes the display of mc ugly when used in a tty console (as opposed to an xterm), but does not affect its performance.
This time any new files are copied to the target directory and the changes to any files I modified are applied to the original backup files. Any files I deleted from my home directory will still be in the target directory (rsync doesn't remove deleted files unless you specifically tell it to). The result is, again, a complete copy of the home chris directory at the moment the rsync command is run, plus any files that have been deleted from any home chris directories.
Recently developed file systems for Linux now support journaling, which allows your system to recover from a crash or interruption easily. The ext3, ReiserFS, and JFS (IBM) file systems maintain a record of file and directory changes, called a journal, which can be used to recover files and directories in use when a system suddenly crashes due to unforeseen events such as power interruptions. Most distributions currently use the ext3 file system as their default, though you also have the option of using ReiserFS or JFS, an independently developed journaling system.
Deletes the specified files or (when the -r option is specified) recursively deletes all subdirectories of the specified files and directories. The -i option causes the command to prompt for confirmation the -f option suppresses confirmation. Because deleted files cannot generally be recovered, the -f option should be used only with extreme care, particularly when used by the root user.
Deleted files are maintained in this salvageable state unless one of the following occurs The server runs out of free space on the disk and begins to overwrite files that have been deleted for a specified period of time. The oldest deleted files are overwritten first. A configurable NSS parameter defines the amount of time a file must remain deleted before it can be overwritten. The administrator sets the NSS parameter immediatePurgeofDeietedFiies. All volumes on that server will immediately purge deleted files. (The default for this parameter is Off. To disable immediate purge, set the NSS parameter NoimmediatePurgeofDeieteFiies.)
U This attribute saves undelete information. This allows a utility to be developed that works with that information to salvage deleted files. Note Although you can use quite a few attributes, you should be aware that most attributes are rather experimental and are of use only if an application is used that can work with the given attribute. For example, it doesn't make sense to apply the u attribute as long as no application has been developed that can use this attribute to recover deleted files.
Search forward for text using slash ( ) or backwards using Escape slash (Esc- ). Type n to search again. Press Tab to jump to the next new or unread message. Or go to the previous one using Esc-Tab. Type s to save the current message to a file. Type d to delete a message and u to undelete it.
This information may also help when recovering deleted files. Imagine that a user tells you that he has created three files, home user filel, home user file2, and home user file3, but has accidentally deleted file2 and desperately needs to get it back. The first thing you can do is use the lsdel command from the debugfs interface. Chances are it will give you a list of deleted inodes, including their original size and deletion time. Listing 5-7 shows an example.
The Midnight Commander can make use of a virtual file system and includes a special undelete file system that can be used on ext2 3 partitions. The mc utility is just an interface to the ext2fs library, and the virtual file system handles for you the nitty-gritty details of the file system that the previous paragraph alluded to. As with any simplified solution to complex issues, mc must make some assumptions for you that might result in less data being recovered than if you use the manual method. For that reason, using mc for recovering deleted files is not a standard method. Be patient because it will take quite a while for the deleted files to be displayed. You will see a list of inodes that you can examine with the text editor (using the F4 key) then use the F12 key to perform a Save As operation, renaming the file to something appropriate for your use. Repeat this process until you have renamed and saved all the files. Press the Shift+F10 key to exit when done. Note that you might...
Tracking down and removing or changing the permissions of a former user's files can prevent confusion or possibly even spurious accusations of wrongdoing in the future. Unless the user was involved in system cracking, there's no reason to think that the user's password will be duplicated in the password database. No system file's ownership or permissions should need changing when deleting a user. Although overwriting deleted files with random data may be useful in some high-security environments or with unusually sensitive data, it's not a necessary practice on most systems. See Chapter 4 for more information.
There are so many tasks that are performed by a system administrator that it is impossible to mention them all. Our duties have included building network cables, installing a network, configuring routers, answering user questions, assembling tables upon which the system equipment will sit, and almost anything else you can think of. A system administrator who appears to have free time is fair game. Never mind that you are compiling a kernel on a remote machine while downloading accidentally deleted files from a backup. Users, managers, salespeople they all think they have the right to interrupt you and start you off on a wholly unexpected task if you aren't obviously doing something already and sometimes even when you are. Life for a system administrator is never boring.
The three icons in the upper-left corner of the desktop are links to your home directory, the system trash can that holds your deleted files until you empty the trash, and the Computer icon that opens the Nautilus graphical shell. The Nautilus File Manager gives you access to your files and directories so you can do typical file management tasks like copying and moving files. In addition to regular file management tasks, the Nautilus File Manager lets you perform desktop management as well. You look more closely at Nautilus in this chapter. Take a closer look at these icons.
Although this may sound like a good idea, passwords that protect hard drives are often only a maximum of 8 bytes and have very small character sets (case-insensitive letters and numbers). These passwords can be brute-forced or even removed using a variety of methods. Several solutions exist for removing passwords, allowing drives to be imaged in a forensically sound manner, and replacing passwords afterward while the machine owner is unaware of the intrusion. Vogon (http www.vogon-international.com), a company specializing in data recovery, data conversion, and investigative services, has developed a password cracker pod specifically for this purpose. This functionality is mainly designed for forensic investigators and law enforcement officers who need covert access to machines, but it can be useful for administrative purposes as well.
Both desktops (Figures 5-1 and 5-2) show icons for your computer, your home folder, and the trash can for deleted files. Both desktops have something similar to the Windows taskbar. On the KDE desktop, the taskbar, called the panel, appears along the bottom of the screen. GNOME has two such panels one on the top and the other on the bottom of the screen. Even though the appearance may look slightly different, the panels serve the same purpose on both KDE and GNOME desktops they provide buttons for accessing menus and starting applications, and they show buttons for any applications you've started (or were automatically started for you).
Once a file is removed, it is permanently deleted and there is no command you can use to restore it you cannot undelete it. (Although if you can unmount the filesystem that contained the file immediately after you deleted the file, a wizard might be able to help reconstruct the lost file by using grep to search the filesystem device file.)
If you've moved to Ubuntu or Kubuntu from a Microsoft Windows system, you may be painfully aware that it is easy to recover files that you've deleted on a Windows system because it initially just erases the directory entry that identifies the file or folder that you're deleting. This is not the case on a Linux system. When you delete a file or directory on a Linux system, all of the disk storage associated with the file, directory, and the contents of that directory are returned to a general list of free space that is available on your system. Although deleted files and directories can still be recovered on a Linux system, it is much harder to do so and requires the assistance of someone who really knows the details of the filesystem.
The menu bar indicates how to mark messages for deletion or undelete them, save messages to a directory, or reply to a message. Type m to compose a new message and it opens your default editor (for me, vi) to create the message. If you want to read your mail without having your fingers leave your keyboard, mutt is a nice choice. (It even handles attachments )
Regardless of how careful we are or how robust our hardware might be, it is highly likely that sometimes data will be lost. Though fatal system problems are rare, accidentally deleted files or mistakes using mv or cp are common. Routine system backup is essential to avoid losing precious data.
Ubuntu's shred can help in both situations. It simply overwrites a file (or hard disk removable storage) over and over again with random data, so that the original data isn't recoverable (even by extremely specialized data recovery agencies, or so it's claimed by shred's creators).
But how do you get rid of files Again, this is relatively easy, but first a word of caution the shell doesn't operate any kind of Recycle Bin. Once a file is deleted, it's gone forever. (There are utilities you can use to recover files, but these are specialized tools and aren't to be relied on for day-to-day use.)
Because a directory is a file as well, the same techniques for file recovery can be used to recover entire directories. The Ext2fs Undeletion of Directory Structures mini-HOWTO (see Reference ) is written as a companion to the Ext2fs Undeletion mini-HOWTOboth of which should be on everyone's required reading list if you want to successfully undelete any files. The manual technique described in the HOWTOs is too long and complex to be covered in this book.
Believe it or not, this method is the most flexible way to back up an Oracle database. Rather than backing up an entire database (as with cold backups), these hot backups save files at the tablespace level. If a disk fails and you lose a mount point, simply restore those lost files from the hot backup. During the recovery, Oracle will notice that they have an earlier timestamp than specified in the surviving control files. Oracle will go into recovery mode and prompt you for all the archive log files after the timestamp of the restored files. Oracle will then replay the transactions in the archived log files and eventually the online redo log files to bring all the data files to the current SCN. Oracle will then open the database for business as normal.
Warning Incremental and differential backups store new files, but most backup programs don't record the fact that deleted files have been deleted. Therefore, a full restore from a complete set of backups may include more files than were stored on the system at any given point in time. (A very new backup program, Duplicity, http www.nongnu.org duplicity , claims to record information on deleted files.) If disk space is tight, or if your users regularly create and delete very large files, be sure to consider this fact when designing a backup plan, lest you run out of disk space when you restore data.
Although current storage technologies, such as RAID, hot-swappable hard drives, and network-attached storage are making servers ever more secure in their capability to maintain data, there are still many ways in which data can be lost or corrupted. For those situations, it is necessary to have a backup of your network data so that lost files can be recovered.
Restoring data from a backup is fairly straightforward when your system is fundamentally intact and you just need to recover a handful of lost files you run the backup software in reverse, as it were. For instance, you use tar's -extract command rather than the -create command. In the case of tar, you must specify the files or directories you want to recover on the command line, or else the system will attempt to recover everything. For instance, you might type these commands to restore the home alice directory from a tape backup if you've accidentally deleted it
The filesystem creation process is inherently destructive. If you accidentally create a filesystem in error, it will be impossible to recover files from the old filesystem unless you're very knowledgeable about filesystem data structures, or you can pay somebody with such knowledge. Recovery costs are apt to be very high.
For data that you need to back up frequently throughout the day, the hard drive can serve you well. Using a hard drive increases the chances of the most recent data recovery. For very critical and not so critical environments where frequent backups and fast recovery are important, use RAID.
Many IT departments' efforts are dedicated to the time-consuming task of repairing end-user desktops. For this task, most IT systems use open-source imaging systems that exist today, such as SystemImager, partimage, FileZilla, clonezilla, Frisbee, rsync, rdiff-backup, ADIOS and so on, or their commercial equivalents, including Norton Ghost, Active, True Image and Image. All these tools create a compressed image of a client's hard drive data and save it in case a future data recovery is necessary. An image is the complete copy of a filesystem, and it usually is stored on a backup server. When image changes are small, incremental backup is used to improve performance.
NFS has undergone several revisions over the years. These NFS version numbers are often appended to the NFS acronym, as in NFSv2 for NFS version 2. This level is the default in the Linux kernel options to use NFSv3 or NFSv4, you must activate extra features, which are visible in Figure 24.1's menu. As of the late 2.4.x and 2.5.x kernels, NFS support through NFSv3 is reasonably stable and complete. NFSv4 support is still considered experimental as of the 2.5.66 kernel (the latest as I write). I recommend avoiding the use of an experimental NFS driver it may result in poor performance, lost files, or other problems.
The options available for badblocks are detailed in the man page. They allow for very low-level manipulation of the file system that is useful for data recovery by file system experts or for file system hacking, but are beyond the scope of this chapter and the average user.
When you create a file, there is one hard link to it. You can then delete the file or, using Linux terminology, remove the link with the rm utility. When you remove the last hard link to a file, you can no longer access the information stored there and the operating system releases the space the file occupied on the disk for subsequent use by other files. This space is released even if symbolic links to the file remain. When there is more than one hard link to a file, you can remove a hard link and still access the file from any remaining link. Unlike DOS and Windows, Linux does not provide an easy way to undelete a file once you have removed it. A skilled hacker, however, can sometimes piece the file together with time and effort.
The menu bar indicates how to mark messages for deletion or undelete them, save messages to a directory, or reply to a message. Type m to compose a new message and it opens your default editor (for me, vi) to create the message. Type y to send the message. If you want to read mail without having your fingers leave your keyboard, mutt is a nice choice. (It even handles attachments )
It's easy to accidentally delete a file. Part of the purpose of backups is to allow recovery of such files, but if the file hadn't been backed up before its accidental deletion, the backup does no good. Therefore, there are undelete utilities available, and certain window managers use a trash can icon by default, which doesn't immediately delete files, instead storing them in a temporary limbo before deleting them. The rm command does not actually erase the deleted file from the disk but instead removes the file's inode or index pointer, leaving the data in place until that same inode is reused. Because the data still exists, it is often possible to retrieve it. While the ext2 filesystem motivated several undeletion packages methods, the new ext3 system is much more stable, so the utilities that were being developed, GtkRecover for instance, are no longer being developed. The command-line version of this, Recover, still exists, although not much development activity is presently going...
The filesystem determines how information is written to your drive. Filesystems are responsible for providing features such as security, data recovery and, of course, performance. If you've been a geek since the days of DOS, you've no doubt noticed that DOS and Windows 95 and its successors seem to have considerably faster disk access than Windows NT and Windows 2000. The FAT filesystem used by DOS and Windows 95 et al is a very simple filesystem which sports the following it's terribly featureless, and it's fast. By contrast, the Windows NT 2000 XP filesystem NTFS provides all manner of features such as security and data recovery, but is clunky and slow. This is the standard payoff between features and speed.
Recently developed file systems for Linux now support journaling, which allows your system to recover from a crash or interruption easily. The ext3, ReiserFS, XFS, and Journaled File System (JFS) from IBM maintain a record of file and directory changes, called a journal, which can be used to recover files and directories in use when a system suddenly crashes due to unforeseen events such as power interruptions. Most distributions currently use the ext3 file system as their default, though you also have the option of using ReiserFS or JFS, an independently developed journaling system.
Keeping a journal entails more work for a file system than any nonjournal method. Though all journaling systems maintain a file system's directory structure (the metadata), they offer various levels of file data recovery. Maintaining file data recovery information can be time-consuming, slowing down the file system's response time. At the same time, journaling systems make more efficient use of the file system, providing a faster response time than the nonjournal ext2 file system.
The ext3 file system maintains full metadata recovery support (directory tree recovery), but it offers various levels of file data recovery. In effect, you are trading off less file data recovery for more speed. The ext3 file system supports three options writeback, ordered, and journal. The default option, writeback, provides only metadata recovery, no file data recovery. The ordered option supports limited file data recovery, and the journal option provides for full file data recovery. Any files in the process of being changed during a crash will be recovered. To specify a ext3 option, use the data option in the mount command
RAID is best suited to desktops and servers that hold multiple hard drives and require data recovery. The most favored form of RAID, RAID 5, requires a minimum of three hard drives. RAID, with the exception of RAID 0, provides the best protection against hard drive failure and is considered a necessity for storage-intensive tasks like enterprise, database, and Internet server operations. It can also provide peace of mind for smaller operations, providing recovery from hard disk failure. Keep in mind that there are different forms of RAID, each with advantages and weaknesses. RAID 0 provides no recovery capabilities at all. After setting up a RAID array, you could then implement LVM volumes on the array.
There is one restriction and recommendation for logical volumes. The boot partition cannot be part of a logical volume. You still have to create a separate hard disk partition as your boot partition with the boot mountpoint in which your kernel and all needed boot files are installed. In addition, it is recommended that you not place your root partition on a logical volume. Doing so can complicate any needed data recovery. This is why a default partition configuration set up during Fedora installation will include a separate boot partition of 100MB of type ext3, whereas the root and swap partitions will be installed on logical volumes. There will be two partitions, one for the logical group (LVM physical volume, pv) holding both swap and root volumes, and another for the boot partition (ext3). The logical volumes will in turn both be ext3 file systems.
Although they appear similar, - -force and - -nodeps serve different purposes. --nodeps only disables dependency checks. Use it only if you are certain that a dependency conflict will not cause problems later on. --force forces package installation regardless of all potential problems except for dependency violations. As a result, some situations may require using - -force and --nodeps together. Common uses of --force include installing an older version of the same package (perhaps because the newer version is too buggy), reinstalling deleted files, and restoring altered files to their pristine state.
If you're the paranoid type and you're still stuck using Windows, you need to get Eraser. Eraser is an open-source security tool for Windows that makes sure deleted files are erased and overwritten completely before they are deleted. Even if you don't buy into the black helicopter scenario, there's no doubt that, at least for a time, your deleted files may still be accessible. That's where Eraser comes in. It overwrites the file with other data before it deletes the file. And, that's not all. Eraser not only overwrites the disk area used by the file, it also actually gets out a knife and scrapes off that part of the disk surface that contained the file (just kidding about that last part).
The particular benefit of a Trash bin on a UNIX-based system is that deleting files is permanent. No Undelete feature is included as part of standard Linux or UNIX. The Trash bin lets you move unwanted files out of your way, but save them for a while in case you need to retrieve them.
Obtaining or creating the software to run the backups unattended. Added features, like a script that e-mails you when the backups have finished, also provide some peace of mind. When the system crashes and there is data that is not contained in a backup, other methods of data recovery are required. Your familiarity with these methods will help you get through troubled times with less frustration. Backups and disaster recovery are covered in Chapter 9, Backing Up and Restoring.
This command accepts many of the same options as cp, In, and mv. Specifically, from Table 5.2, -f, -i, and -r work with rm. Unlike some operating systems' file-deletion tools, rm is permanent Linux doesn't store deleted files in any sort of trash can folder. Chapter 12 provides pointers to tools and utilities you can use to recover deleted files or to implement a holding area to prevent files from being immediately deleted.
To restore or undelete a file, click the Trash icon and a window will open up displaying its contents (see Figure 4-26). You can then drag and drop the files shown to other places in the computer's filesystem, or right-click any item for a list of further features, including options to permanently delete or restore a file. There is also an Empty Trash button in this window.
The rm command, especially with the -rf parameter, is very dangerous. It recursively removes an entire directory without stopping to verify any of the files. Run as the root user, this has the potential to really cause problems on your system. Be very careful and make sure you are erasing what you mean to erase. There is no undelete command.
Debugfs The debugfs utility allows you to open the ext2 file system debugger. From this debugger, you can perform powerful tasks. To do this, some internal commands are available from the file system debugger. One of them is the lsdel command, which will list files that were recently deleted from your file system. After finding the inodes of these recently deleted files, you can use the dump command, followed by the number of the inode. For example, use dump somefile to dump everything the inode refers to in the file somefile that is created automatically. You must be aware, however, that this works only if you are acting quickly. When a file is removed on a Linux file system, the inode and blocks that were used by the file are flagged as available, and the next time data is written to the volume, they can be overwritten. Also, you should be aware of on disadvantage of the debugfs method it doesn't know anything about file or directory names. Therefore, you can see the inode number of...
Sizeable backups can be split into several files that together make the original backup. A backup file can be split for burning to removable media or saving on ftp-server (data recovery directly from ftp-server requires the archive to be split into files no more than 2 Gb in size).
If you do experience data loss, it is sometimes possible to recover that data using the filesystem maintenance tools described in Section 6.1.5 in Chapter 6. Unlike some other operating systems, however, it's generally not possible to undelete a file that has been removed by rm or overwritten by a careless cp or mv command (for example, copying one file over another destroys the file to which you're copying). In these extreme cases, backups are key to recovering from problems.
The Acronis One-Click Restore is a minimal addition to your rescue media, allowing one-click data recovery from an image archive, stored on this media. This means that at boot from the media and clicking restore all data will be silently restored to the original place. No options or selections like resizing partitions will be possible.
Data recovery directly from ftp-server requires the archive to be split into files no more than 2 Gb in size. If you suppose that some of the files may be larger, first copy the entire archive (along with the initial full backup) to a local hard disk or network share disk.
Undelete utilities for Linux are few and far between. The Linux philosophy is that users shouldn't delete files they really don't want to delete, and if they do, they should be restored from backups. Nonetheless, in a pinch there are some tricks you can use to try to recover accidentally deleted files. One of these tricks is the recover utility, which is headquartered at and available with most Linux distributions. Unfortunately, this tool has several drawbacks. The first is that it was designed for ext2fs, and so it doesn't work with most journaling filesystems. (It may work with ext3fs, though.) Another problem is that recover takes a long time to do anything, even on small partitions. I frequently see network programs such as web browsers and mail clients crash when recover runs. Finally, in my experience, recover frequently fails to work at all if you type recover dev sda4, for instance, to recover files from dev sda4, the program may churn for a while, consume a lot of CPU time,...
Everything you might have been told about the rm command isn't entirely true, and by the end of this article, you'll find that Linux does have an undelete of sorts. there is no way you can get it back. Undelete commands were for DOS users anyway we Linux users knew better, right Well, it turns out, we don't. Most Linux users I know have deleted the wrong files at least once in their lives. Now, the best protection against this is a backup (noticing a common thread in this series ), but if you don't have a backup, you aren't completely without hope. Everything you might have been told about the rm command isn't entirely true, and by the end of this article, you'll find that Linux does have an undelete of sorts. It turns out that accident-prone Linux users aren't the only ones who want to recover deleted files. In fact, deleted file recovery is particularly useful for forensics, as attackers might try to delete files to cover their tracks. Forensics tools work with the filesystem on a...
Linux manages the hard disk and user spaces in RAM, using i-node tables to maintain the disk information and a memory manager for user information. Linux writes any changes to the i-node tables to the disk drive every so often, but it maintains the RAM copies as the most recent because of RAM's greater speed. If you shut down the power before Linux writes any changes to the disk, the disk contents and the i-node tables written on the disk may not match, causing lost files and an incorrect list of what disk space is available. Even worse, if Linux was in the process of writing the i-node table or any other information at the moment the power is turned off, the write process is interrupted, and disk head crashes or bad sector information can result. The same principle applies to any processes that are running. If, for example, you were running a database reindex when you killed the power, the indexes and databases may be corrupted. Shutting down the Linux system properly makes sure that...
Two external data recovery agencies were sent copies of the HP-UX dd output file and staff at NCC's ESCROW department, ex-CHC employees also assisted, but a solution to the problem could not be found. Hex dumps of the tapes suggested that they had been created on a DEC Alpha running OSF-1 and various Internet Newsgroups were contacted. Through this medium, a kind person in the United States offered to read the dd file on his DEC Alpha, but this also failed. Eric Taylor, The Walton Centre's UNIX Guru, was assigned the case. Eric discovered from the hex dumps that the files stored were clearly not streamed. He then tried for a few weeks to work out the data structure used as he was certain it was a file system of some kind, but without any indication what file system it was, this proved to be an impossible task. At this point what was needed was a bit of luck
If you wish to gather more information, the tct (The Coroner's Toolkit from Dan Farmer and Wietse Venema) package contains utilities which perform a post mortem analysis of a system. tct allows the user to collect information about deleted files, running processes and more. See the included documentation for more information. These same utilities and some others can be found in Sleuthkit and Autopsy (http www.sleuthkit.org ) by Brian Carrier, which provides a web front-end for forensic analysis of disk images. In Debian you can find both sleuthkit (the tools) and autopsy (the graphical front-end).
|WiseRecovery Data Recovery Software|
Computer Hard Drive Data Recovery
Learn How To Recover Your Hard Drive Data After A Computer Failure.