I have a directory full of .txt files. I need to copy these files into a new directory, but I want to merge the files that have a duplicate line inside of the text. For example, these files would be merged into one file when copied to the new directory because the first line of text is the same. file1.txt My favorite food: ice cream I love ice cream because it cools me off on a hot day. file2.txt My favorite food: ice cream I really like ice cream because it is sweet and creamy. Any help on how to write a command for this is appreciated.
Hi To combine the text files with duplicate initial lines into one file in the new directory, run the following command in a Unix-based terminal: Code: mkdir new_directory && awk 'BEGIN{a=0} /^My favorite food:/{if(!a[$0]++)c++;print>>"new_directory/file"c".txt"}' directory/*.txt This programme will create a new directory named new directory and then use awk to scan all the.txt files in the old directory for lines that begin with "My favourite food:" (replace directory with the actual name of your directory). If the line is unique, it will be added to a new file in the new directory called file1.txt. If the line has previously been viewed, it will be included in the following file (file2.txt, file3.txt, etc.). After executing this command, you should have a new directory containing merged files with distinct initial lines. Hope that will help
