My Centos is terribly slow

Discussion in 'Server Operation' started by Galled, Dec 3, 2010.

  1. Galled

    Galled New Member

    Hello I have a Centos 5:

    Linux xxx.xxx.xxx 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:19:18 EDT 2009 i686 i686 i386 GNU/Linux

    Problems:

    - When I connect with SSH, each command take until 2 min to run!
    - Restart the server takes 16 minutes on average.
    - I have a web page in this server (PHP (Joomla) with Apache/2.2.8 (Unix) ) and each page takes 1 min in load.
    - I assumed it was because originally it was connected with IPv6, because when running the command "lsmod | grep v6" showed me the following:

    ipv6 261473 125
    xfrm_nalgo 13381 1 ipv6

    But using this website (http://techno.mayix.net/deshabilitar-ipv6-en-redhatcentos/), I disable IPv6 connection because when running the command "lsmod | grep v6" showed me nothing. But the problems of slowness, still.
    - When I run the command top give me this:
    Code:
    top - 09:40:08 up 10 min,  1 user,  load average: 18.89, 14.99, 8.02
    Tasks: 153 total,  13 running, 140 sleeping,   0 stopped,   0 zombie
    Cpu(s): 57.1%us, 42.4%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.2%hi,  0.3%si,  0.0%st
    Mem:   1555380k total,   667200k used,   888180k free,    18332k buffers
    Swap:  2048276k total,        0k used,  2048276k free,   193784k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                              
     2265 mysql     15   0  133m  23m 6052 S 35.8  1.5   2:43.58 mysqld                                                                                                               
     2868 apache    15   0 42092  21m 4520 S 18.2  1.4   0:19.51 httpd                                                                                                                
     3112 apache    16   0 41844  21m 4536 S 18.2  1.4   0:09.18 httpd                                                                                                                
     2506 apache    15   0 42108  21m 4516 S 17.9  1.4   0:23.15 httpd                                                                                                                
     3471 apache    16   0 41732  20m 4252 S 17.9  1.4   0:02.59 httpd                                                                                                                
     2567 apache    15   0 42060  21m 4496 S 16.9  1.4   0:20.84 httpd                                                                                                                
     2625 apache    16   0 41840  21m 4568 R 16.9  1.4   0:16.53 httpd                                                                                                                
     3461 apache    15   0 41460  20m 4260 S 16.6  1.4   0:02.17 httpd                                                                                                                
     2593 apache    15   0 42084  21m 4480 S 16.3  1.4   0:15.37 httpd                                                                                                                
     2429 apache    15   0 42040  21m 4484 S  9.8  1.4   0:14.31 httpd                                                                                                                
     3477 apache    18   0 36900  15m 3792 R  5.5  1.0   0:01.24 httpd                                                                                                                
     3499 apache    19   0 37432  16m 3840 R  3.3  1.1   0:01.40 httpd                                                                                                                
     2417 apache    15   0 40544  20m 4508 S  1.3  1.3   0:18.89 httpd                                                                                                                
     3520 root      15   0  2420 1060  800 R  1.3  0.1   0:00.46 top                                                                                                                  
     3464 apache    15   0 41332  20m 4432 S  0.7  1.4   0:04.03 httpd                                                                                                                
     2423 operador  15   0 10032 1596 1024 S  0.3  0.1   0:00.54 sshd                                                                                                                 
     2463 root      16   0  2052  636  556 S  0.3  0.0   0:01.84 hald-addon-stor                                                                                                      
     2482 apache    15   0 37224  16m 4508 S  0.3  1.1   0:17.95 httpd                                                                                                                
        1 root      15   0  2152  672  580 S  0.0  0.0   0:02.09 init                                                                                                                 
        2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.10 migration/0                                                                                                          
        3 root      34  19     0    0    0 R  0.0  0.0   0:00.00 ksoftirqd/0                                                                                                          
        4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0                                                                                                           
        5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.05 migration/1                                                                                                          
        6 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/1                                                                                                          
        7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/1                                                                                                           
        8 root      10  -5     0    0    0 S  0.0  0.0   0:00.06 events/0                                                                                                             
        9 root      10  -5     0    0    0 S  0.0  0.0   0:00.04 events/1                                                                                                             
       10 root      10  -5     0    0    0 S  0.0  0.0   0:00.03 khelper                                                                                                              
       11 root      14  -5     0    0    0 S  0.0  0.0   0:00.02 kthread                                                                                                              
       15 root      10  -5     0    0    0 S  0.0  0.0   0:00.08 kblockd/0                                                                                                            
       16 root      10  -5     0    0    0 S  0.0  0.0   0:00.05 kblockd/1                                                                                                            
       17 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 kacpid                                                                                                               
       74 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/0                                                                                                             
       75 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/1                                                                                                             
       78 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 khubd                                                                                                                
       80 root      14  -5     0    0    0 S  0.0  0.0   0:00.03 kseriod                                                                                                              
      146 root      19   0     0    0    0 S  0.0  0.0   0:00.00 pdflush                                                                                                              
      147 root      15   0     0    0    0 S  0.0  0.0   0:00.50 pdflush                                                                                                              
      148 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 kswapd0                                                                                                              
      149 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 aio/0                                                                                                                
      150 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 aio/1                                                                                                                
      305 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 kpsmoused                                                                                                            
      349 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 mpt_poll_0                                                                                                           
      350 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 scsi_eh_0                                                                                                            
      354 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 ata/0                                                                                                                
      355 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 ata/1                                                                                                                
      356 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 ata_aux                                                                                                              
      363 root      13  -5     0    0    0 S  0.0  0.0   0:00.00 kstriped                                                                                                             
      376 root      13  -5     0    0    0 S  0.0  0.0   0:00.00 ksnapd               
    
    Why is this server so slow? What could be happening? Why Apache appear to have many instances created? And why always consumes so much CPU?

    P.D.: Sorry for my english
     
    Last edited: Dec 3, 2010
  2. mini14

    mini14 Member

    Those Apache children are serving up webpages..otherwise they would not show at the top of your output while running "Top"

    Have you checked your access and error logs to see if there's any suspicious activity?

    What hardware is in this box? It seems to be overloaded and IMO should not be even with that level of activity. If you notice, the percent of CPU usage adds up to quite a bit more than 100%
     
  3. Galled

    Galled New Member

    "dmesg" showed me this:
    Code:
    Linux version 2.6.18-128.1.6.el5 ([email protected]) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Wed Apr 1 09:19:18 EDT 2009
    BIOS-provided physical RAM map:
    BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
    BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
    BIOS-e820: 00000000000ca000 - 00000000000cc000 (reserved)
    BIOS-e820: 00000000000dc000 - 00000000000e0000 (reserved)
    BIOS-e820: 00000000000e4000 - 0000000000100000 (reserved)
    BIOS-e820: 0000000000100000 - 000000005fef0000 (usable)
    BIOS-e820: 000000005fef0000 - 000000005feff000 (ACPI data)
    BIOS-e820: 000000005feff000 - 000000005ff00000 (ACPI NVS)
    BIOS-e820: 000000005ff00000 - 0000000060000000 (usable)
    BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
    BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
    BIOS-e820: 00000000fffe0000 - 0000000100000000 (reserved)
    640MB HIGHMEM available.
    896MB LOWMEM available.
    found SMP MP-table at 000f6aa0
    Memory for crash kernel (0x0 to 0x0) notwithin permissible range
    disabling kdump
    Using x86 segment limits to approximate NX protection
    On node 0 totalpages: 393216
    DMA zone: 4096 pages, LIFO batch:0
    Normal zone: 225280 pages, LIFO batch:31
    HighMem zone: 163840 pages, LIFO batch:31
    DMI present.
    Using APIC driver default
    ACPI: RSDP (v002 PTLTD ) @ 0x000f6a30
    ACPI: XSDT (v001 INTEL 440BX 0x06040000 VMW 0x01324272) @ 0x5fefa138
    ACPI: FADT (v004 INTEL 440BX 0x06040000 PTL 0x000f4240) @ 0x5fefee98
    ACPI: BOOT (v001 PTLTD $SBFTBL$ 0x06040000 LTP 0x00000001) @ 0x5fefa202
    ACPI: MADT (v001 PTLTD APIC 0x06040000 LTP 0x00000000) @ 0x5fefa1a4
    ACPI: DSDT (v001 PTLTD Custom 0x06040000 MSFT 0x03000001) @ 0x00000000
    ACPI: PM-Timer IO Port: 0x1008
    ACPI: Local APIC address 0xfee00000
    ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
    Processor #0 6:15 APIC version 17
    ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
    Processor #1 6:15 APIC version 17
    ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
    ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
    ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
    IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
    ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
    ACPI: IRQ0 used by override.
    ACPI: IRQ2 used by override.
    ACPI: IRQ9 used by override.
    Enabling APIC mode: Flat. Using 1 I/O APICs
    Using ACPI (MADT) for SMP configuration information
    Allocating PCI resources starting at 70000000 (gap: 60000000:9ec00000)
    Detected 1861.345 MHz processor.
    Built 1 zonelists. Total pages: 393216
    Kernel command line: ro root=LABEL=/
    mapped APIC to ffffd000 (fee00000)
    mapped IOAPIC to ffffc000 (fec00000)
    
    Enabling fast FPU save and restore... done.
    Enabling unmasked SIMD FPU exception support... done.
    Initializing CPU#0
    CPU 0 irqstacks, hard=c0754000 soft=c0734000
    PID hash table entries: 4096 (order: 12, 16384 bytes)
    Console: colour VGA+ 80x25
    Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
    Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
    Memory: 1551684k/1572864k available (2122k kernel code, 19808k reserved, 883k data, 228k init, 655296k highmem)
    Checking if this processor honours the WP bit even in supervisor mode... Ok.
    Calibrating delay using timer specific routine.. 3727.42 BogoMIPS (lpj=1863714)
    Security Framework v1.0.0 initialized
    SELinux: Initializing.
    SELinux: Starting in permissive mode
    selinux_register_security: Registering secondary module capability
    Capability LSM initialized as secondary
    Mount-cache hash table entries: 512
    CPU: After generic identify, caps: 0febfbff 20100000 00000000 00000000 00002201 00000000 00000001
    CPU: After vendor identify, caps: 0febfbff 20100000 00000000 00000000 00002201 00000000 00000001
    CPU: L1 I cache: 32K, L1 D cache: 32K
    CPU: L2 cache: 4096K
    CPU: After all inits, caps: 0febf3ff 20100000 00000000 00000940 00002201 00000000 00000001
    Intel machine check architecture supported.
    Intel machine check reporting enabled on CPU#0.
    Checking 'hlt' instruction... OK.
    SMP alternatives: switching to UP code
    ACPI: Core revision 20060707
    CPU0: Intel(R) Xeon(R) CPU E5320 @ 1.86GHz stepping 07
    SMP alternatives: switching to SMP code
    Booting processor 1/1 eip 3000
    CPU 1 irqstacks, hard=c0755000 soft=c0735000
    Initializing CPU#1
    Calibrating delay using timer specific routine.. 3724.65 BogoMIPS (lpj=1862325)
    CPU: After generic identify, caps: 0febfbff 20100000 00000000 00000000 00002201 00000000 00000001
    CPU: After vendor identify, caps: 0febfbff 20100000 00000000 00000000 00002201 00000000 00000001
    CPU: L1 I cache: 32K, L1 D cache: 32K
    CPU: L2 cache: 4096K
    CPU: After all inits, caps: 0febf3ff 20100000 00000000 00000940 00002201 00000000 00000001
    Intel machine check architecture supported.
    Intel machine check reporting enabled on CPU#1.
    CPU1: Intel(R) Xeon(R) CPU E5320 @ 1.86GHz stepping 07
    Total of 2 processors activated (7452.07 BogoMIPS).
    ENABLING IO-APIC IRQs
    ..TIMER: vector=0x31 apic1=0 pin1=2 apic2=-1 pin2=-1
    checking TSC synchronization across 2 CPUs: passed.
    Brought up 2 CPUs
    sizeof(vma)=84 bytes
    sizeof(page)=32 bytes
    sizeof(inode)=340 bytes
    sizeof(dentry)=136 bytes
    sizeof(ext3inode)=492 bytes
    sizeof(buffer_head)=52 bytes
    sizeof(skbuff)=172 bytes
    migration_cost=4860
    
    checking if image is initramfs... it is
    Freeing initrd memory: 2546k freed
    NET: Registered protocol family 16
    No dock devices found.
    ACPI: bus type pci registered
    PCI: PCI BIOS revision 2.10 entry at 0xfd8a0, last bus=1
    PCI: Using configuration type 1
    Setting up standard PCI resources
    ACPI: Interpreter enabled
    ACPI: Using IOAPIC for interrupt routing
    ACPI: PCI Root Bridge [PCI0] (0000:00)
    PCI quirk: region 1000-103f claimed by PIIX4 ACPI
    PCI quirk: region 1040-104f claimed by PIIX4 SMB
    Boot video device is 0000:00:0f.0
    ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
    ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10 11 14 15) *0, disabled.
    ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *9 10 11 14 15)
    ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 *11 14 15)
    ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 9 *10 11 14 15)
    Linux Plug and Play Support v0.97 (c) Adam Belay
    pnp: PnP ACPI init
    pnp: PnP ACPI: found 12 devices
    usbcore: registered new driver usbfs
    usbcore: registered new driver hub
    PCI: Using ACPI for IRQ routing
    PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report
    NetLabel: Initializing
    NetLabel: domain hash size = 128
    NetLabel: protocols = UNLABELED CIPSOv4
    NetLabel: unlabeled traffic allowed by default
    PCI: Bridge: 0000:00:01.0
    IO window: disabled.
    MEM window: disabled.
    PREFETCH window: disabled.
    PCI: Setting latency timer of device 0000:00:01.0 to 64
    NET: Registered protocol family 2
    IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
    TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
    TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
    TCP: Hash tables configured (established 131072 bind 65536)
    TCP reno registered
    Simple Boot Flag at 0x36 set to 0x80
    apm: BIOS version 1.2 Flags 0x03 (Driver version 1.16ac)
    apm: disabled - APM is not SMP safe.
    audit: initializing netlink socket (disabled)
    
    type=2000 audit(1291385607.018:1): initialized
    highmem bounce pool size: 64 pages
    Total HugeTLB memory allocated, 0
    VFS: Disk quotas dquot_6.5.1
    Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
    SELinux: Registering netfilter hooks
    Initializing Cryptographic API
    alg: No test for crc32c (crc32c-generic)
    ksign: Installing public key data
    Loading keyring
    - Added public key 88A2B9299D40C296
    - User ID: CentOS (Kernel Module GPG key)
    io scheduler noop registered
    io scheduler anticipatory registered
    io scheduler deadline registered
    io scheduler cfq registered (default)
    Limiting direct PCI/PCI transfers.
    pci_hotplug: PCI Hot Plug PCI Core version: 0.5
    ACPI: Processor [CPU0] (supports 8 throttling states)
    ACPI: Processor [CPU1] (supports 8 throttling states)
    Real Time Clock Driver v1.12ac
    Non-volatile memory driver v1.2
    Linux agpgart interface v0.101 (c) Dave Jones
    agpgart: Detected an Intel 440BX Chipset.
    agpgart: AGP aperture is 256M @ 0x0
    Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
    serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
    serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
    00:09: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
    00:0a: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
    RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize
    Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
    ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
    PIIX4: IDE controller at PCI slot 0000:00:07.1
    PIIX4: chipset revision 1
    PIIX4: not 100% native mode: will probe irqs later
    ide1: BM-DMA at 0x1068-0x106f, BIOS settings: hdc:DMA, hdd:pio
    Probing IDE interface ide1...
    hdc: VMware Virtual IDE CDROM Drive, ATAPI CD/DVD-ROM drive
    ide1 at 0x170-0x177,0x376 on irq 15
    Probing IDE interface ide0...
    ide-floppy driver 0.99.newide
    usbcore: registered new driver hiddev
    usbcore: registered new driver usbhid
    drivers/usb/input/hid-core.c: v2.6:USB HID core driver
    PNP: PS/2 Controller [PNP0303:KBC,PNP0f13:MOUS] at 0x60,0x64 irq 1,12
    serio: i8042 KBD port at 0x60,0x64 irq 1
    serio: i8042 AUX port at 0x60,0x64 irq 12
    mice: PS/2 mouse device common for all mice
    md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
    md: bitmap version 4.39
    input: AT Translated Set 2 keyboard as /class/input/input0
    TCP bic registered
    
    Initializing IPsec netlink socket
    NET: Registered protocol family 1
    NET: Registered protocol family 17
    Using IPI No-Shortcut mode
    Time: tsc clocksource has been installed.
    ACPI: (supports S0 S1 S4 S5)
    input: ImPS/2 Generic Wheel Mouse as /class/input/input1
    Freeing unused kernel memory: 228k freed
    Write protecting the kernel read-only data: 397k
    ohci_hcd: 2005 April 22 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
    USB Universal Host Controller Interface driver v3.0
    SCSI subsystem initialized
    Fusion MPT base driver 3.04.07
    Copyright (c) 1999-2008 LSI Corporation
    Fusion MPT SPI Host driver 3.04.07
    ACPI: PCI Interrupt 0000:00:10.0[A] -> GSI 17 (level, low) -> IRQ 169
    mptbase: ioc0: Initiating bringup
    ioc0: LSI53C1030 B0: Capabilities={Initiator}
    scsi0 : ioc0: LSI53C1030 B0, FwRev=00000000h, Ports=1, MaxQ=128, IRQ=169
    Vendor: VMware, Model: VMware Virtual S Rev: 1.0 
    Type: Direct-Access ANSI SCSI revision: 02
    target0:0:0: Beginning Domain Validation
    target0:0:0: Domain Validation skipping write tests
    target0:0:0: Ending Domain Validation
    target0:0:0: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127)
    SCSI device sda: 16777216 512-byte hdwr sectors (8590 MB)
    sda: Write Protect is off
    sda: Mode Sense: 5d 00 00 00
    sda: cache data unavailable
    sda: assuming drive cache: write through
    SCSI device sda: 16777216 512-byte hdwr sectors (8590 MB)
    sda: Write Protect is off
    sda: Mode Sense: 5d 00 00 00
    sda: cache data unavailable
    sda: assuming drive cache: write through
    sda: sda1 sda2 sda3
    sd 0:0:0:0: Attached scsi disk sda
    Vendor: VMware, Model: VMware Virtual S Rev: 1.0 
    Type: Direct-Access ANSI SCSI revision: 02
    target0:0:1: Beginning Domain Validation
    target0:0:1: Domain Validation skipping write tests
    target0:0:1: Ending Domain Validation
    target0:0:1: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127)
    SCSI device sdb: 67108864 512-byte hdwr sectors (34360 MB)
    sdb: Write Protect is off
    sdb: Mode Sense: 5d 00 00 00
    sdb: cache data unavailable
    sdb: assuming drive cache: write through
    SCSI device sdb: 67108864 512-byte hdwr sectors (34360 MB)
    sdb: Write Protect is off
    sdb: Mode Sense: 5d 00 00 00
    sdb: cache data unavailable
    sdb: assuming drive cache: write through
    
    sdb: sdb1
    sd 0:0:1:0: Attached scsi disk sdb
    libata version 3.00 loaded.
    device-mapper: uevent: version 1.0.3
    device-mapper: ioctl: 4.11.5-ioctl (2007-12-12) initialised: [email][email protected][/email]
    device-mapper: dm-raid45: initialized v0.2429
    kjournald starting. Commit interval 5 seconds
    EXT3-fs: mounted filesystem with ordered data mode.
    SELinux: Disabled at runtime.
    SELinux: Unregistering netfilter hooks
    type=1404 audit(1291385636.454:2): selinux=0 auid=4294967295 ses=4294967295
    parport: PnPBIOS parport detected.
    parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
    hdc: ATAPI 1X CD-ROM drive, 32kB Cache, UDMA(33)
    Uniform CD-ROM driver Revision: 3.20
    input: PC Speaker as /class/input/input2
    VMware vmxnet virtual NIC driver
    ACPI: PCI Interrupt 0000:00:11.0[A] -> GSI 18 (level, low) -> IRQ 177
    Found vmxnet/PCI at 0x1424, irq 177.
    features:
    numRxBuffers = 100, numRxBuffers2 = 1
    ACPI: PCI Interrupt 0000:00:12.0[A] -> GSI 19 (level, low) -> IRQ 185
    Found vmxnet/PCI at 0x14a4, irq 185.
    features:
    numRxBuffers = 100, numRxBuffers2 = 1
    sd 0:0:0:0: Attached scsi generic sg0 type 0
    sd 0:0:1:0: Attached scsi generic sg1 type 0
    piix4_smbus 0000:00:07.3: Found 0000:00:07.3 device
    piix4_smbus 0000:00:07.3: Host SMBus controller not enabled!
    pcnet32.c:v1.32 18.Mar.2006 [email][email protected][/email]
    Floppy drive(s): fd0 is 1.44M
    FDC 0 is a post-1991 82077
    lp0: using parport0 (interrupt-driven).
    lp0: console ready
    ACPI: AC Adapter [ACAD] (on-line)
    ACPI: Power Button (FF) [PWRF]
    md: Autodetecting RAID arrays.
    md: autorun ...
    md: ... autorun DONE.
    device-mapper: multipath: version 1.0.5 loaded
    EXT3 FS on sda3, internal journal
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on sda1, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-0, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    VMware hgfs: HGFS is disabled in the host
    Adding 2048276k swap on /dev/sda2. Priority:-1 extents:1 across:2048276k
    IA-32 Microcode Update Driver: v1.14a <[email protected]>
    VMware hgfs: HGFS is disabled in the host
    VMware memory control driver initialized
    
    vmmemctl: started kernel thread pid=1840
    VMCI: Major device number is: 253
    vsock: no version for "VMCIMemcpyToQueueV" found: kernel tainted.
    VMware hgfs: HGFS is disabled in the host
    set_rtc_mmss: can't update from 0 to 57
    set_rtc_mmss: can't update from 0 to 58
    set_rtc_mmss: can't update from 0 to 58
    set_rtc_mmss: can't update from 0 to 58
    set_rtc_mmss: can't update from 0 to 58
    set_rtc_mmss: can't update from 0 to 58
    set_rtc_mmss: can't update from 1 to 58
    set_rtc_mmss: can't update from 1 to 58
    set_rtc_mmss: can't update from 1 to 58
    set_rtc_mmss: can't update from 1 to 58
    set_rtc_mmss: can't update from 1 to 58
    set_rtc_mmss: can't update from 1 to 58
    set_rtc_mmss: can't update from 2 to 58
    set_rtc_mmss: can't update from 2 to 58
    set_rtc_mmss: can't update from 2 to 58
    set_rtc_mmss: can't update from 2 to 58
    set_rtc_mmss: can't update from 2 to 58
    set_rtc_mmss: can't update from 2 to 58
    set_rtc_mmss: can't update from 2 to 58
    set_rtc_mmss: can't update from 2 to 58
    set_rtc_mmss: can't update from 2 to 58
    set_rtc_mmss: can't update from 2 to 58
    set_rtc_mmss: can't update from 3 to 59
    set_rtc_mmss: can't update from 3 to 59
    set_rtc_mmss: can't update from 3 to 59
    set_rtc_mmss: can't update from 3 to 59
    set_rtc_mmss: can't update from 3 to 59
    set_rtc_mmss: can't update from 3 to 59
    set_rtc_mmss: can't update from 3 to 59
    set_rtc_mmss: can't update from 4 to 59
    set_rtc_mmss: can't update from 4 to 59
    set_rtc_mmss: can't update from 4 to 59
    set_rtc_mmss: can't update from 4 to 59
    set_rtc_mmss: can't update from 4 to 59
    set_rtc_mmss: can't update from 4 to 59
    set_rtc_mmss: can't update from 4 to 59
    set_rtc_mmss: can't update from 4 to 59
    set_rtc_mmss: can't update from 5 to 59
    set_rtc_mmss: can't update from 5 to 59
    set_rtc_mmss: can't update from 5 to 59
    set_rtc_mmss: can't update from 5 to 59
    set_rtc_mmss: can't update from 5 to 59
    set_rtc_mmss: can't update from 5 to 59
    set_rtc_mmss: can't update from 5 to 59
    
    Does it serve?
     
    Last edited: Dec 3, 2010
  4. mini14

    mini14 Member

    I would guess that one of three things are happening:

    Your hardware has a major problem

    Your software is improperly configured

    Your server is under some sort of attack

    Have you checked the following for clues?
    /var/log/messages
    /var/log/secure
    /var/log/httpd/ (both access and error logs in there)
     
  5. Galled

    Galled New Member

    In "cat /var/log/messages" I have some rare messages:

    Code:
    Dec  3 08:28:18 prodsinfa avahi-daemon[2368]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns!
    ...
    ...
    Dec  3 08:29:22 myserve smartd[2623]: Device: /dev/sda, opened 
    Dec  3 08:29:22 myserve smartd[2623]: Device: /dev/sda, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sda' to turn on SMART features 
    Dec  3 08:29:22 myserve smartd[2623]: Unable to register SCSI device /dev/sda at line 32 of file /etc/smartd.conf 
    Dec  3 08:29:22 myserve smartd[2623]: Device /dev/sda not available 
    Dec  3 08:29:22 myserve smartd[2623]: Device: /dev/sdb, opened 
    Dec  3 08:29:22 myserve smartd[2623]: Device: /dev/sdb, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdb' to turn on SMART features 
    Dec  3 08:29:22 myserve smartd[2623]: Unable to register SCSI device /dev/sdb at line 33 of file /etc/smartd.conf 
    Dec  3 08:29:22 myserve smartd[2623]: Device /dev/sdb not available 
    Dec  3 08:29:22 myserve smartd[2623]: Monitoring 0 ATA and 0 SCSI devices 

    I do not know if this will help but i have:

    df -h showed me this:
    Code:
    Filesystem            Size  Used Avail Use% Mounted on
    /dev/sda3             5.8G  2.2G  3.4G  39% /
    /dev/sda1              99M   22M   73M  23% /boot
    /dev/mapper/gv1-vl1    30G   22G  6.9G  76% /var
    tmpfs                 760M     0  760M   0% /dev/shm
    
     
  6. Galled

    Galled New Member

    I do not know if this will help but i have:

    In "/etc/httpd/error_log" I have this:

    Code:
    [Fri Dec 03 16:26:36 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/
    [Fri Dec 03 16:26:37 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:26:41 2010] [error] [client yyy.yy.yyy.yyy] File does not exist: /var/www/html/iconos, referer: http://www.google.com.pe/imgres?imgurl=http://www.mydomain1.com/iconos/rrnn_mapa6.jpg&imgrefurl=http://www.skyscrapercity.com/showthread.php%3Ft%3D334498%26page%3D11&usg=__TE78OCO71_qWPUzsZbnIk7NHi5Y=&h=1113&w=780&sz=192&hl=es&start=8&zoom=1&tbnid=LcPaP_LYt3SzUM:&tbnh=150&tbnw=105&prev=/images%3Fq%3Dmapa%2Bde%2Blas%2Becorregiones%2Bdel%2Bperu%26um%3D1%26hl%3Des%26sa%3DX%26biw%3D1276%26bih%3D574%26tbs%3Disch:1&um=1&itbs=1
    [Fri Dec 03 16:26:41 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:26:41 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/
    [Fri Dec 03 16:26:42 2010] [error] [client yyy.yy.yyy.yyy] File does not exist: /var/www/html/iconos, referer: http://www.google.com.pe/imgres?imgurl=http://www.mydomain1.com/iconos/rrnn_mapa6.jpg&imgrefurl=http://www.skyscrapercity.com/showthread.php%3Ft%3D334498%26page%3D11&usg=__TE78OCO71_qWPUzsZbnIk7NHi5Y=&h=1113&w=780&sz=192&hl=es&start=8&zoom=1&tbnid=LcPaP_LYt3SzUM:&tbnh=150&tbnw=105&prev=/images%3Fq%3Dmapa%2Bde%2Blas%2Becorregiones%2Bdel%2Bperu%26um%3D1%26hl%3Des%26sa%3DX%26biw%3D1276%26bih%3D574%26tbs%3Disch:1&um=1&itbs=1
    [Fri Dec 03 16:26:47 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=salida
    [Fri Dec 03 16:26:55 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:26:55 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:26:56 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=salida
    [Fri Dec 03 16:26:57 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=salida
    [Fri Dec 03 16:26:57 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=salida
    [Fri Dec 03 16:26:57 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:26:59 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:27:01 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=salida
    [Fri Dec 03 16:27:01 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=salida
    [Fri Dec 03 16:27:06 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:27:07 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:27:10 2010] [error] [client zzz.zzz.zzz.zz] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=salida
    [Fri Dec 03 16:27:10 2010] [error] [client zzz.zzz.zzz.zz] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=salida
    [Fri Dec 03 16:27:15 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=salida
    [Fri Dec 03 16:27:15 2010] [error] [client aa.a.aa.aaa] ModSecurity: Warning. Match of "rx ^OPTIONS$" against "REQUEST_METHOD" required. [file "/etc/httpd/modsecurity.d/modsecurity_crs_21_protocol_anomalies.conf"] [line "41"] [id "960015"] [msg "Request Missing an Accept Header"] [severity "CRITICAL"] [tag "PROTOCOL_VIOLATION/MISSING_HEADER"] [hostname "www.mydomain.com"] [uri "/compress/css.php"] [unique_id "M81kEX8AAAEAADJwui8AAAAN"]
    [Fri Dec 03 16:27:15 2010] [error] [client aa.a.aa.aaa] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/etc/httpd/modsecurity.d/modsecurity_crs_21_protocol_anomalies.conf"] [line "48"] [id "960009"] [msg "Request Missing a User Agent Header"] [severity "WARNING"] [tag "PROTOCOL_VIOLATION/MISSING_HEADER"] [hostname "www.mydomain.com"] [uri "/compress/css.php"] [unique_id "M81kEX8AAAEAADJwui8AAAAN"]
    [Fri Dec 03 16:27:16 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=salida
    [Fri Dec 03 16:27:18 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:27:18 2010] [error] [client xxx.xx.xx.xx] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:27:19 2010] [error] [client ddd.jjj.vvv.tt] ModSecurity: Warning. Match of "rx ^OPTIONS$" against "REQUEST_METHOD" required. [file "/etc/httpd/modsecurity.d/modsecurity_crs_21_protocol_anomalies.conf"] [line "41"] [id "960015"] [msg "Request Missing an Accept Header"] [severity "CRITICAL"] [tag "PROTOCOL_VIOLATION/MISSING_HEADER"] [hostname "www.mydomain.com"] [uri "/frontpage/rss.html"] [unique_id "NAq8an8AAAEAADJ9RlUAAAAO"]
    [Fri Dec 03 16:27:20 2010] [error] [client kkk.cc.vvv.ii] File does not exist: /var/www/html/templates/mydomaininterioresbyairgat/images/bg_patrias.png, referer: http://www.mydomain.com/K0mPr355/CsS.php?id=interiores_airgat
    [Fri Dec 03 16:27:26 2010] [error] [client zzz.zzz.zzz.zz] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:27:28 2010] [error] [client zzz.zzz.zzz.zz] File does not exist: /var/www/html/images/logo.jpg, referer: http://frenteweb.mydomain.com/mywebsystem/?mod=consulta_cult
    [Fri Dec 03 16:27:48 2010] [error] [client kkk.jjj.ii.ttt] File does not exist: /var/www/html/templates/mydomaininterioresbyairgat/images/bg_patrias.png, referer: http://www.mydomain.com/redes-de-contacto/directorio-telefonico.html
    [Fri Dec 03 16:27:52 2010] [error] [client bbb.bb.bbb.bb] File does not exist: /var/www/html/templates/mydomaininterioresbyairgat/images/bg_patrias.png, referer: http://www.mydomain.com/K0mPr355/CsS.php?id=interiores_airgat
    [Fri Dec 03 16:28:55 2010] [error] [client bbb.bb.bbb.bb] File does not exist: /var/www/html/images/logo.jpg, referer: http://aguaje.mydomain2.com/mywebsystem/
    [Fri Dec 03 16:28:55 2010] [error] [client bbb.bb.bbb.bb] File does not exist: /var/www/html/templates/mydomain_baella, referer: http://aguaje.mydomain2.com/mywebsystem/
    
    In "free -m" I have this:

    Code:
                 total       used       free     shared    buffers     cached
    Mem:          1518        876        642          0         47        566
    -/+ buffers/cache:        263       1255
    Swap:         2000          0       2000
    
     
  7. falko

    falko Super Moderator Howtoforge Staff

    Is this a physical system or a virtual machine?
     
  8. Galled

    Galled New Member

    Initially I assumed it was a real machine, but then appears to be a "VMWare" but do not yet know the version.
     
  9. Galled

    Galled New Member

    Ok.... Mmmm. This is so weird. I just conect with my server by SSH and now it's fast!

    It is possible that a requirement to note the speed of the server after changing the settings by disabling the "IPv6" is restarting the client? Because that's all I've done and now the site loads fast and the connection via SSH is fast too!

    Now "top" showed me this:
    Code:
    top - 10:58:14 up 2 days,  9:10,  1 user,  load average: 0.56, 1.04, 0.64
    Tasks: 124 total,   2 running, 121 sleeping,   0 stopped,   1 zombie
    Cpu(s):  0.3%us,  0.7%sy,  0.0%ni, 99.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
    Mem:   1555380k total,  1460660k used,    94720k free,    60656k buffers
    Swap:  2048276k total,       80k used,  2048196k free,  1114088k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                            
     4299 apache    15   0     0    0    0 Z  0.7  0.0   0:01.09 httpd <defunct>                                                    
     4513 apache    15   0 38092  17m 4452 S  0.3  1.2   0:01.27 httpd                                                              
     4646 root      15   0  2412 1040  800 R  0.3  0.1   0:00.45 top                                                                
        1 root      15   0  2152  640  552 S  0.0  0.0   0:01.24 init                                                               
        2 root      RT  -5     0    0    0 S  0.0  0.0   0:01.49 migration/0                                                        
        3 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/0                                                        
        4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0                                                         
        5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.78 migration/1                                                        
        6 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/1                                                        
        7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/1                                                         
        8 root      10  -5     0    0    0 S  0.0  0.0   0:00.32 events/0                                                           
        9 root      10  -5     0    0    0 S  0.0  0.0   0:00.15 events/1                                                           
       10 root      10  -5     0    0    0 S  0.0  0.0   0:00.25 khelper                                                            
       11 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kthread                                                            
       15 root      19  -5     0    0    0 S  0.0  0.0   0:01.12 kblockd/0                                                          
       16 root      10  -5     0    0    0 S  0.0  0.0   0:00.67 kblockd/1                                                          
       17 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 kacpid                                                             
       74 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/0                                                           
       75 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/1                                                           
       78 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 khubd                                                              
       80 root      18  -5     0    0    0 S  0.0  0.0   0:00.00 kseriod                                                            
      146 root      15   0     0    0    0 S  0.0  0.0   0:02.54 pdflush                                                            
      147 root      15   0     0    0    0 S  0.0  0.0   0:15.59 pdflush                                                            
      148 root      10  -5     0    0    0 S  0.0  0.0   0:03.60 kswapd0                                                            
      149 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 aio/0                                                              
      150 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 aio/1                                                              
      304 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 kpsmoused                                                          
      349 root      10  -5     0    0    0 S  0.0  0.0   0:00.01 mpt_poll_0                                                         
      350 root      16  -5     0    0    0 S  0.0  0.0   0:00.00 scsi_eh_0                                                          
      354 root      16  -5     0    0    0 S  0.0  0.0   0:00.00 ata/0                                                              
      355 root      16  -5     0    0    0 S  0.0  0.0   0:00.00 ata/1                                                              
      356 root      16  -5     0    0    0 S  0.0  0.0   0:00.00 ata_aux                                                            
      363 root      18  -5     0    0    0 S  0.0  0.0   0:00.00 kstriped                                                           
      376 root      19  -5     0    0    0 S  0.0  0.0   0:00.00 ksnapd                                                             
      387 root      20   0 10032 2752 2204 S  0.0  0.2   0:00.03 sshd                                                               
      389 root      10  -5     0    0    0 S  0.0  0.0   0:04.76 kjournald                                                          
      394 operador  15   0 10032 1596 1024 S  0.0  0.1   0:00.04 sshd                                                               
      397 operador  16   0  4620 1368 1156 S  0.0  0.1   0:00.04 bash                                                               
      414 root      10  -5     0    0    0 S  0.0  0.0   0:00.34 kauditd    
    
    And "df -h" showed me this:
    Code:
    Filesystem            Size  Used Avail Use% Mounted on
    /dev/sda3             5.8G  2.2G  3.4G  39% /
    /dev/sda1              99M   22M   73M  23% /boot
    /dev/mapper/gv1-vl1    30G   21G  7.1G  75% /var
    tmpfs                 760M     0  760M   0% /dev/shm
    
    And in "free -m" I have this
    Code:
                 total       used       free     shared    buffers     cached
    Mem:          1518       1454         64          0         59       1089
    -/+ buffers/cache:        304       1213
    Swap:         2000          0       2000
    
    As far as I know, since the friday nobody has touched the server and the only thing I did was simply restart my pc.

    I will continue to monitor the server during the day to verify that the problem is solved.

    Thank you very much for all your help and your time!

    P.D.:I have reviewed the "history" and an administrator has restarted the "Apache" and "MySQL". I supose he has made a modify of some config file of this services, but in the "history" trail does show nothing similar to "vi" or "gedit". He may have done this change by FTP or another way.
     
    Last edited: Dec 6, 2010
  10. Galled

    Galled New Member

    I talk to an administrator. He has replaced the entire virtual machine Centos 32-bit to a "Ubuntu Sever 10.10" and has installed the apache in worker mode and fast-cgi php. He has made in the weekend.

    Bone in the end the solution was to change the OS. XD
     
  11. atul_spmail

    atul_spmail New Member

    Even i am facing similar issues, i even tried using new clone against old server. still speed drops down at 10:30 pm and around 12:30 am. Solutino i try is to just restart mysql and httpd services
    Do any one has permanent fix for this
     

    Attached Files:

  12. atul_spmail

    atul_spmail New Member

    can anyone suggest on my TOP command output
     

    Attached Files:

Share This Page