remove duplicate lines in a file

Discussion in 'Programming/Scripts' started by jack777, Apr 25, 2008.

  1. jack777

    jack777 New Member

    Hi
    Having a file as follows
    Code:
    ==================================================
    Date           Time         error code      server 
    ======================================================
    [B]12-29-08 [/B]   [B]10:10 [/B]       121221          [B]server A[/B]
    [B]12-29-08 [/B]   [B]10:12 [/B]       121221          [B]server A[/B]
    12-29-08    10:10        121221          server B
    12-29-08    10:10        121221          server c
    
    Need to remove the duplicate lines with the following conditions

    if date=12-29-08 and server=server A .
    Must delete the line with old time and keep the latest.

    anyone can suggest

    Thanks in advance
     
  2. burschik

    burschik New Member

    Not really duplicate

    #!/bin/awk

    BEGIN { pattern = /^[[:digit:]]+-[[:digit:]]+-[[:digit:]]/; }

    $1 !~ pattern { print; }

    $1 ~ pattern {
    if ($1 != previous_date && previous_date != "" ||
    $5 != previous_server && previous_server != "") {
    print previous_line;
    }

    previous_date = $1;
    previous_server = $5;
    previous_line = $0;
    }

    END { print previous_line; }
     

Share This Page