User Tools

Site Tools


experimenting:splitting

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

experimenting:splitting [Wednesday, 29 August 2007 : 15:12:02] (current)
Line 1: Line 1:
 +====== Splitting output to several files ======
 +When you perform different measurements,​ you often want to put different measurements into different files.
 +Since Chi 1.0 only allows output with ''​!!'',​ you have no choice but to output all measurements to the screen (or file, if you use [[#​Capturing output|capturing output]]).
  
 +For example, you output starting and stopping times of a machine:
 +<code chi>
 +|[ var x: lot
 +:: *( a?x        // get a new product
 +    ; !! "MACH: start ", time, "​\n"​
 +    ; delay 3.0  // perform processing
 +    ; !! "MACH: stop ", time, "​\n"​
 +    ; b!x        // output the product to the next process
 +    )
 +]|
 +</​code>​
 +
 +In a WIP controller, you output the WIP on each update:
 +<code chi>
 +|[ var wip: nat = 0
 +:: *( !! "WIP: ", wip, "​\t",​ time "​\n"​
 +    ; ( wip < maxwip -> g! ; wip := wip + 1
 +      | e? ; wip := wip - 1
 +      )
 +    )
 +]|
 +</​code>​
 +The ''​wip''​ variable keeps track of the amounts of products in the line.
 +Synchronization with the generator ''​g!''​ means that a new product enters the line, synchronization with the exit process ''​e?''​ means that a product has left the line.
 +At each change, the wip and the time is printed.
 +
 +When you combine the above fragments in one model, output is generated both by the machine and by the WIP controller.
 +Before further processing can be done, you want to split the output into different files, one file with machine output and one file with WIP controller output.
 +There are two ways to do this, by using ''​grep'',​ and (more advanced) by using ''​sed''​.
 +
 +Both use the uniq prefixes printed at the beginning of each line ("''​MACH:''"​ and "''​WIP:''"​).
 +
 +===== Splitting output using ''​grep''​ =====
 +The ''​grep''​ command reads a file, and outputs all lines that match some pattern.
 +
 +You can use this to split output of your simulation by first capturing all output to a file (see [[#​Capturing output|capturing output]] above), followed by runng a grep for each prefix.
 +For example
 +<​code>​
 +$ grep MACH: myfile > mach_output
 +</​code>​
 +reads the file ''​myfile'',​ and outputs all lines that have ''​MACH:''​ in them. The matching lines are output to screen, but in the example, the output is captured to the ''​mach_output''​ file.
 +
 +Grep is very fast, and quite powerful. Full details can be found in the online manual (type ''​man grep''​).
 +The disadvantage of this approach is that you first need to capture the simulation output in a (often big) file, then run the grep for each prefix of your model.
 +
 +===== Splitting output using ''​sed''​ =====
 +
 +Sed is much more powerful than grep. "​sed"​ is short hand for "​stream editor",​ it is a kind of automated editing program.
 +It reads input, and based on patterns that match, it adds, removes, or replaces text. Finally, the result is output.
 +Full details can be found in the online manual of sed (type ''​man sed''​).
 +
 +Splitting output into different files is a simple job for sed. Several solutions exist, three of them are shown here, each of them more advanced than the previous solution.
 +With each solution, you should first write an '​editing script'​ (or rather '​splitting script'​),​ and use that to process the output with ''​sed''​.
 +
 +The first solution is much like several grep's together. Put the following lines in ''​sedfile1'':​
 +<​code>​
 +/^MACH:/ w mach.txt
 +/^WIP:/ w wip.txt
 +d
 +</​code>​
 +For each file that you want to create, you write the prefix between ''"/​^"''​ and ''"/"'',​ and the filename after the "​w"​.
 +Run the script like
 +<​code>​
 +$ startmodel mymodel | sed -f sedfile1
 +</​code>​
 +If you previously captured the simulation output in the ''​my_output''​ file, you can use
 +<​code>​
 +$ cat my_output | sed -f sedfile1
 +</​code>​
 +In both cases, lines that start with "​MACH:"​ are put in the file ''​mach.txt'',​ and lines that start with "​WIP:"​ are put in the file ''​wip.txt''​.
 +
 +In the editing script, ''"/"''​ characters define the start and the end of the pattern that must be matched, the ''"​^"''​ means only to look for matches at the start of the line (thus a line like ''"​blabla MACH: WIP:"''​ wil not match). After the pattern, the "​w"​ (write) command means that the matching line should be written to a file. The filename is put after the "​w"​ command.
 +The "​d"​ at the last line deletes all lines, so you don't get them all at your screen.
 +
 +The prefix ("​MACH:"​ or "​WIP:"​) is copied to the output file. If you don't want that, you can delete it with a ''​s''​ (substitute) command:
 +<​code>​
 +s/^MACH: // w mach.txt
 +s/^WIP: // w wip.txt
 +d
 +</​code>​
 +As before, the pattern to match is between the first and the second "/"​ characters. The text to replace it with is between the second and third "/"​ characters. Here, the matching prefix is thus replaced with nothing (ie it is deleted). Note the additional " " before the second "/"​. In this way, you also delete the space character immediately after the ":"​.
 +
 +After the substitution,​ the line is written (without prefix) to the file specified by the "​w"​ command.
 +
 +----
 +
 +
 +The disadvantage of the first solution is that all lines that have no match are silently discarded due to the "​d"​ (delete) command at the end.
 +You can fix this by deleting lines that match. At the end, you are then left with lines that do not match.
 +Make a file ''​sedfile2''​ like:
 +<​code>​
 +/^MACH:/ { w mach.txt
 +           d
 +         }
 +/​^WIP:/ ​ { w wip.txt
 +           d
 +         }
 +</​code>​
 +You can run the script in the same way as above.
 +
 +This script has the same patterns as before. When a line matches, all the commands between the curly brackets are executed. In this case, the first command ("​w"​) writes the line to a file. The second command ("​d"​) deletes the line. In other words, a line that starts with a prefix is written and then deleted.
 +Since the final "​d"​ line is missing (below the last pattern), lines that never match are not deleted. Instead, they are output to the screen (and they can be captured with a ''>''​ if you want).
 +
 +As before, you can also remove the "​MACH:"​ or "​WIP:"​ prefix. Change the script to
 +<​code>​
 +/^MACH:/ { s/^MACH: // w mach.txt
 +           d
 +         }
 +/​^WIP:/ ​ { s/^WIP: // w wip.txt
 +           d
 +         }
 +</​code>​
experimenting/splitting.txt · Last modified: Wednesday, 29 August 2007 : 15:12:02 (external edit)