Benutzer:Rdiez/BrittleOperatingSystems - Versionsgeschichte

Rdiez: Weiterleitung nach Benutzer:Rdiez/PageRemoved erstellt

2015-05-15T19:28:34Z

Weiterleitung nach Benutzer:Rdiez/PageRemoved erstellt

Rdiez: OOM Killer

2014-09-05T19:20:28Z

OOM Killer

Rdiez am 5. September 2014 um 19:14 Uhr

2014-09-05T19:14:23Z

Rdiez: USB delays

2014-09-05T19:09:55Z

USB delays

Rdiez: Linux cache, filesystem semantics

2014-08-27T20:12:46Z

Linux cache, filesystem semantics

Rdiez am 13. Dezember 2013 um 20:46 Uhr

2013-12-13T20:46:06Z

Rdiez am 13. Dezember 2013 um 20:17 Uhr

2013-12-13T20:17:13Z

Rdiez am 13. Dezember 2013 um 20:05 Uhr

2013-12-13T20:05:10Z

Rdiez am 13. Dezember 2013 um 19:22 Uhr

2013-12-13T19:22:00Z

Rdiez am 13. Dezember 2013 um 18:52 Uhr

2013-12-13T18:52:01Z

@@ Zeile 1: / Zeile 1: @@
-{{BenutzerSeitenNichtVeraendernWarnung|rdiez}}
+#REDIRECT [[Benutzer:Rdiez/PageRemoved]]
-−
-= Brittle Operating Systems =
-−
-== Rendering Your System Unresponsive Is Really Easy ==
-−
-It's hard to believe how brittle modern Operating Systems still are. After years of development and refinement, you would expect most mainstream OSes to have reached a reasonable level of robustness and security. However, a simple [http://en.wikipedia.org/wiki/Fork_bomb fork bomb] easily brings most OSes down.
-−
-It is as if the concept of [http://en.wikipedia.org/wiki/Denial-of-service_attack denial of service] would not belong in the [http://en.wikipedia.org/wiki/Software_security software security] category. Apparently, [http://en.wikipedia.org/wiki/Responsiveness responsiveness] does not fit into the OS writers' idea of [http://en.wikipedia.org/wiki/Robustness_%28computer_science%29 software robustness] either.
-−
-Admittedly, the simplest standard ''fork bomb'' does not do the trick any more, as most OSes nowadays limit the maximum number of child processes per session. However, it takes about a minute to write an enhanced version that still works, here it is:
-−
- #!/bin/bash
-−
- ENABLE_BOMB=true
- ENABLE_PRINT_LEN=false
-−
- TEST="ab"
-−
- bomb() {
-−
-   if $ENABLE_BOMB; then
-     bomb &
-   fi
-−
-   TEST="$TEST$TEST"
-−
-   echo $TEST >/dev/null
-   echo >/dev/null
-−
-   if $ENABLE_PRINT_LEN; then
-     echo "TEST string length: ${#TEST}"
-   fi
-−
-   bomb
- };
-−
- bomb
-−
-I left a little extra code inside so that you can comfortably experiment with it. Just save it to a file called "ForkBomb-WARNING-may-crash-your-computer.sh", start it as follows and watch your OS slowly grind to a halt:
-−
- bash ForkBomb-WARNING-may-crash-your-computer.sh
-−
-That's it. You don't need to be logged on as [http://en.wikipedia.org/wiki/Root_user superuser] or change any special system configuration setting.
-−
-Your system may not technically die, it will probably start [http://en.wikipedia.org/wiki/Thrashing_%28computer_science%29 thrashing] so much that it quickly becomes unusable. On the other hand, some systems become unstable when they run out of memory. The OS kernel itself may be robust enough to cope, but you need many other system services in order to use your computer effectively, and chances are that at least one of them will start to fail and may not recover even if enough memory becomes available later on.
-−
-I have seen that little script bring the following systems down:
-−
-* Microsoft Windows Vista (under Cygwin). <br/> You get a low-memory warning which does not really help to deal with the problem. If you don't wait too long, you can log out and after a while you will get your system back.
-* Apple OS X. <br/> Some things still worked, like the moving window animations or the system menu, but nothing else really worked. Pressing Ctrl+C in the terminal window seems to stop the script but does not actually help. Trying to log out has no effect either.
-* Kubuntu 13.10. <br/> Tested on a Konsole window. The mouse pointers starts to jerk around and within 5 seconds the computer is frozen solid. None of the usual key combinations works any more.
-−
-Please do [[Benutzer:rdiez|drop me a line]] if you try it on other OSes, so that I can update the list above. Did you find one that resists the script in its default configuration?
-−
-The ''fork bomb'' scenario is not as uncommon as you may think. The reason I decided to write this post is that I have recently managed to inadvertently render my PC sluggish or unusable in the most simple ways:
-−
-* I clicked on a 1-GiB binary file on a default Kubuntu 13.10 installation. <br/> By default, a single mouse click in KDE opens the file in the Kate editor, which apparently tries to load the whole file at once into RAM. That's enough to bring your KDE desktop down. I normally reconfigure KDE to double-click in order to open a file, but I forgot this time. Kate needs 3,2 GiB of RAM to open a 1,GiB file. Other editors like emacs do not load all contents at once or ask if you really want to open such a large file.
-−
-* I started a build process with "make -j", without specifying a number for the "-j" argument. [http://lists.gnu.org/archive/html/bug-make/2012-02/msg00031.html I reported this problem on the GNU Make mailing list] but found no resonance. [http://stackoverflow.com/questions/9306293/gnu-make-j-causes-high-cpu-load-and-desktop-no-response Other people have realised too]. I know this should be fixed at OS level, but it shouldn't be so easy to make such a mistake with a standard build tool.
-−
-* Just copying a few large files around on traditional hard disks makes most systems pause for seconds at a time or react very slowly to mouse clicks. This is not just because of poor disk prioritisation, see the section about the filesystem cache below for more information.
-−
-* Writing to a USB stick can make a Linux system hang for minutes. See LWN article [http://lwn.net/Articles/572911/ The pernicious USB-stick stall problem] for details.
-−
-* If a big process crashes and generates a coredump, your machine may hang for a long time. David Woodhouse [https://plus.google.com/+DavidWoodhouse/posts/bAaqz2iq6Lk has described this problem].
-−
-Such "crashes" are a pain, as there is no definitive error message and it may not be immediately obvious why your system stopped responding. I just hate wasting time because of simple issues like these!
-−
-Sometimes, the Linux Out Of Memory (OOM) Killer helps keep your system responsive, but at the cost of killing a random process (hopefully the right one). When it does, you do not get a deskop notification about what process got killed and why, which is not really helpful.
-−
-If you keep bringing your PC down in those ways, the only thing that ultimately helps is some system-level limiter like [http://en.wikipedia.org/wiki/Cgroups cgroups] under Linux. What I do not understand is why that's not the default configuration on most systems. Such a control system must surely cost some performance, but with the kind of processing power available today, surely that's preferable to risking a system-wide crash. Besides, I had a quick look and ''cgroups'' and the like do not seem easy enough for the casual user.
-−
-== The Linux Filesystem Cache is Braindead ==
-−
-Say you have 2 GiB of RAM and you copy 2 GiB's worth of data from one disk directory to another. That will effectively flush the Linux filesystem cache, and you don't even have to be root. Anything you want to do afterwards will have to reload any other files needed from disk, which means that the system will always respond slowly after copying large files.
-−
-The Linux cache subsystem does not realise that you are copying a set of large files just once, that they would not fit in the cache anyway, and that other often-used files should remain in the cache instead. Tool ''nocache'' is supposed to come to the rescue, but it's too brutal, it just turns any caching off for the given operation. As a result, if you copy a large number of small files, it will take forever.
-−
-For more information about the Linux cache behaviour, check out [https://bugzilla.samba.org/show_bug.cgi?id=9560 rsync bug 9560 drop-cache option].
-−
-I suspect this issue is not limited to Linux though. Please drop me a line if you know what other OSes are affected.
-−
-== Broken Filesystem Semantics ==
-−
-The standard file system semantics are too simple, they are actually broken for most applications.
-−
-Consider the typical scenario where an application reads a 100 KiB configuration file and overwrites it with 200 KiB's worth of data. If writing the second half fails, the file ends up corrupt. Therefore, before pressing Ctrl+C, think about it twice. The next time you start your favourite application, it may not work any more.
-−
-A typical software work-around is to write the updated configuration to another file, delete the old file and rename the new one. But then, if the renaming fails, or if you press Ctrl+C between the deleting and the renaming, there will be no configuration file next time the application starts. On some systems, you can atomically replace a file with the ''rename'' syscall, but even then there are usually gotchas and exceptions. Note that filesystems like ext4 offer certain degree of protection for file metadata changes, but usually not for the file contents, so you should call ''fsync'' on the new file before renaming it.
-−
-A more robust and portable software work-around is to rotate the configuration files, for example by appending a monotonically-increasing number. The application should regularly delete the old files, or they will forever grow in number. There should be a symbolic link with a well-defined name. Otherwise, any other tool (or the user) will have to deal with changing configuration filenames. When a new configuration file version is created, the link should be updated, but I am not certain whether updating a symbolic link can be done in an atomic operation as far as the underlying filesystem is concerned.
-−
-As you can see, reliably updating a simple configuration file can actually be a tricky operation on today's systems. I would expect to find ready-made libraries for this task, with bindings for all possible scripting languages and support for the most common OSes, but none springs to mind. Many applications seem to rely on good luck when updating files on disk. To make things worse, implementing a software remedy on top of a standard filesystem carries a noticeable performance hit.
-−
-The proper solution to this problem is to use file sytem transactions. Windows Vista and later versions do implement them (see [http://en.wikipedia.org/wiki/Transactional_NTFS Transactional NTFS]), but very few people know that they exist. Besides, Microsoft has stated that the associated TxF API may not be available in future versions of Windows. For simple configuration files, see Win32 APIs ''ReplaceFile'' and ''MoveFileEx'', which, interestingly enough, are not clearly documented as being atomic operations (!).

← Nächstältere Version		Version vom 5. September 2014, 19:20 Uhr
Zeile 67:		Zeile 67:

	Such "crashes" are a pain, as there is no definitive error message and it may not be immediately obvious why your system stopped responding. I just hate wasting time because of simple issues like these!		Such "crashes" are a pain, as there is no definitive error message and it may not be immediately obvious why your system stopped responding. I just hate wasting time because of simple issues like these!
		+
		+	Sometimes, the Linux Out Of Memory (OOM) Killer helps keep your system responsive, but at the cost of killing a random process (hopefully the right one). When it does, you do not get a deskop notification about what process got killed and why, which is not really helpful.

	If you keep bringing your PC down in those ways, the only thing that ultimately helps is some system-level limiter like [http://en.wikipedia.org/wiki/Cgroups cgroups] under Linux. What I do not understand is why that's not the default configuration on most systems. Such a control system must surely cost some performance, but with the kind of processing power available today, surely that's preferable to risking a system-wide crash. Besides, I had a quick look and ''cgroups'' and the like do not seem easy enough for the casual user.		If you keep bringing your PC down in those ways, the only thing that ultimately helps is some system-level limiter like [http://en.wikipedia.org/wiki/Cgroups cgroups] under Linux. What I do not understand is why that's not the default configuration on most systems. Such a control system must surely cost some performance, but with the kind of processing power available today, surely that's preferable to risking a system-wide crash. Besides, I had a quick look and ''cgroups'' and the like do not seem easy enough for the casual user.

← Nächstältere Version		Version vom 5. September 2014, 19:14 Uhr
Zeile 63:		Zeile 63:

	* Writing to a USB stick can make a Linux system hang for minutes. See LWN article [http://lwn.net/Articles/572911/ The pernicious USB-stick stall problem] for details.		* Writing to a USB stick can make a Linux system hang for minutes. See LWN article [http://lwn.net/Articles/572911/ The pernicious USB-stick stall problem] for details.
		+
		+	* If a big process crashes and generates a coredump, your machine may hang for a long time. David Woodhouse [https://plus.google.com/+DavidWoodhouse/posts/bAaqz2iq6Lk has described this problem].

	Such "crashes" are a pain, as there is no definitive error message and it may not be immediately obvious why your system stopped responding. I just hate wasting time because of simple issues like these!		Such "crashes" are a pain, as there is no definitive error message and it may not be immediately obvious why your system stopped responding. I just hate wasting time because of simple issues like these!

← Nächstältere Version		Version vom 5. September 2014, 19:09 Uhr
Zeile 61:		Zeile 61:

	* Just copying a few large files around on traditional hard disks makes most systems pause for seconds at a time or react very slowly to mouse clicks. This is not just because of poor disk prioritisation, see the section about the filesystem cache below for more information.		* Just copying a few large files around on traditional hard disks makes most systems pause for seconds at a time or react very slowly to mouse clicks. This is not just because of poor disk prioritisation, see the section about the filesystem cache below for more information.
		+
		+	* Writing to a USB stick can make a Linux system hang for minutes. See LWN article [http://lwn.net/Articles/572911/ The pernicious USB-stick stall problem] for details.

	Such "crashes" are a pain, as there is no definitive error message and it may not be immediately obvious why your system stopped responding. I just hate wasting time because of simple issues like these!		Such "crashes" are a pain, as there is no definitive error message and it may not be immediately obvious why your system stopped responding. I just hate wasting time because of simple issues like these!

@@ Zeile 2: / Zeile 2: @@
 = Brittle Operating Systems =
 It's hard to believe how brittle modern Operating Systems still are. After years of development and refinement, you would expect most mainstream OSes to have reached a reasonable level of robustness and security. However, a simple [http://en.wikipedia.org/wiki/Fork_bomb fork bomb] easily brings most OSes down.
@@ Zeile 58: / Zeile 60: @@
 * I started a build process with "make -j", without specifying a number for the "-j" argument. [http://lists.gnu.org/archive/html/bug-make/2012-02/msg00031.html I reported this problem on the GNU Make mailing list] but found no resonance. [http://stackoverflow.com/questions/9306293/gnu-make-j-causes-high-cpu-load-and-desktop-no-response Other people have realised too]. I know this should be fixed at OS level, but it shouldn't be so easy to make such a mistake with a standard build tool.
-* Just copying a few large files around on traditional hard disks makes most systems pause for seconds at a time or react very slowly to mouse clicks.
+* Just copying a few large files around on traditional hard disks makes most systems pause for seconds at a time or react very slowly to mouse clicks. This is not just because of poor disk prioritisation, see the section about the filesystem cache below for more information.
 Such "crashes" are a pain, as there is no definitive error message and it may not be immediately obvious why your system stopped responding. I just hate wasting time because of simple issues like these!
 If you keep bringing your PC down in those ways, the only thing that ultimately helps is some system-level limiter like [http://en.wikipedia.org/wiki/Cgroups cgroups] under Linux. What I do not understand is why that's not the default configuration on most systems. Such a control system must surely cost some performance, but with the kind of processing power available today, surely that's preferable to risking a system-wide crash. Besides, I had a quick look and ''cgroups'' and the like do not seem easy enough for the casual user.

← Nächstältere Version		Version vom 13. Dezember 2013, 20:46 Uhr
Zeile 52:		Zeile 52:
	Please do [[Benutzer:rdiez\|drop me a line]] if you try it on other OSes, so that I can update the list above. Did you find one that resists the script in its default configuration?		Please do [[Benutzer:rdiez\|drop me a line]] if you try it on other OSes, so that I can update the list above. Did you find one that resists the script in its default configuration?

−	If you keep bringing your PC down in those ways, the only thing that ultimately helps is some system-level limiter like [http://en.wikipedia.org/wiki/Cgroups cgroups] under Linux. What I do not understand is why that's not the default configuration on most systems. Such a control system must surely cost some performance, but with the kind of processing power available today, surely that's preferable to risking a system-wide crash.	+	The ''fork bomb'' scenario is not as uncommon as you may think. The reason I decided to write this post is that I have recently managed to inadvertently render my PC sluggish or unusable in the most simple ways:
		+
		+	* I clicked on a 1-GiB binary file on a default Kubuntu 13.10 installation. <br/> By default, a single mouse click in KDE opens the file in the Kate editor, which apparently tries to load the whole file at once into RAM. That's enough to bring your KDE desktop down. I normally reconfigure KDE to double-click in order to open a file, but I forgot this time. Kate needs 3,2 GiB of RAM to open a 1,GiB file. Other editors like emacs do not load all contents at once or ask if you really want to open such a large file.
		+
		+	* I started a build process with "make -j", without specifying a number for the "-j" argument. [http://lists.gnu.org/archive/html/bug-make/2012-02/msg00031.html I reported this problem on the GNU Make mailing list] but found no resonance. [http://stackoverflow.com/questions/9306293/gnu-make-j-causes-high-cpu-load-and-desktop-no-response Other people have realised too]. I know this should be fixed at OS level, but it shouldn't be so easy to make such a mistake with a standard build tool.
		+
		+	* Just copying a few large files around on traditional hard disks makes most systems pause for seconds at a time or react very slowly to mouse clicks.
		+
		+	Such "crashes" are a pain, as there is no definitive error message and it may not be immediately obvious why your system stopped responding. I just hate wasting time because of simple issues like these!
		+
		+	If you keep bringing your PC down in those ways, the only thing that ultimately helps is some system-level limiter like [http://en.wikipedia.org/wiki/Cgroups cgroups] under Linux. What I do not understand is why that's not the default configuration on most systems. Such a control system must surely cost some performance, but with the kind of processing power available today, surely that's preferable to risking a system-wide crash. Besides, I had a quick look and ''cgroups'' and the like do not seem easy enough for the casual user.

← Nächstältere Version		Version vom 13. Dezember 2013, 20:17 Uhr
Zeile 51:		Zeile 51:

	Please do [[Benutzer:rdiez\|drop me a line]] if you try it on other OSes, so that I can update the list above. Did you find one that resists the script in its default configuration?		Please do [[Benutzer:rdiez\|drop me a line]] if you try it on other OSes, so that I can update the list above. Did you find one that resists the script in its default configuration?
		+
		+	If you keep bringing your PC down in those ways, the only thing that ultimately helps is some system-level limiter like [http://en.wikipedia.org/wiki/Cgroups cgroups] under Linux. What I do not understand is why that's not the default configuration on most systems. Such a control system must surely cost some performance, but with the kind of processing power available today, surely that's preferable to risking a system-wide crash.

← Nächstältere Version		Version vom 13. Dezember 2013, 19:22 Uhr
Zeile 6:		Zeile 6:

	It is as if the concept of [http://en.wikipedia.org/wiki/Denial-of-service_attack denial of service] would not belong in the [http://en.wikipedia.org/wiki/Software_security software security] category. Apparently, [http://en.wikipedia.org/wiki/Responsiveness responsiveness] does not fit into the OS writers' idea of [http://en.wikipedia.org/wiki/Robustness_%28computer_science%29 software robustness] either.		It is as if the concept of [http://en.wikipedia.org/wiki/Denial-of-service_attack denial of service] would not belong in the [http://en.wikipedia.org/wiki/Software_security software security] category. Apparently, [http://en.wikipedia.org/wiki/Responsiveness responsiveness] does not fit into the OS writers' idea of [http://en.wikipedia.org/wiki/Robustness_%28computer_science%29 software robustness] either.
		+
		+	Admittedly, the simplest standard ''fork bomb'' does not do the trick any more, as most OSes nowadays limit the maximum number of child processes per session. However, it takes about a minute to write an enhanced version that still works, here it is:
		+
		+	#!/bin/bash
		+
		+	ENABLE_BOMB=true
		+	ENABLE_PRINT_LEN=false
		+
		+	TEST="ab"
		+
		+	bomb() {
		+
		+	if $ENABLE_BOMB; then
		+	bomb &
		+	fi
		+
		+	TEST="$TEST$TEST"
		+
		+	echo $TEST >/dev/null
		+	echo >/dev/null
		+
		+	if $ENABLE_PRINT_LEN; then
		+	echo "TEST string length: ${#TEST}"
		+	fi
		+
		+	bomb
		+	};
		+
		+	bomb
		+
		+	I left a little extra code inside so that you can comfortably experiment with it. Just save it to a file called "ForkBomb-WARNING-may-crash-your-computer.sh", start it as follows and watch your OS slowly grind to a halt:
		+
		+	bash ForkBomb-WARNING-may-crash-your-computer.sh
		+
		+	That's it. You don't need to be root or change any special system configuration setting.
		+
		+	I have seen that little script bring the following systems down:
		+
		+	* Apple OS X
		+	* Microsoft Windows Vista (under Cygwin)
		+
		+	Please do [[Benutzer:rdiez\|drop me a line]] if you try it on other OSes, so that I can update the list above. Did you find one that resists the script in its default configuration?