Issue with character sets (debian 3.1)

I've got a really weird problem. I run an oscommerce estore that's in swedish and exported an sql file through phpmyadmin from my webhost. The character set of the sql file is utf-8. The tables I exported the sql are all latin1 and when I open up the file on my ubuntu system all looks just fine.

When i transfer the sql to my debian server and open it, some of the swedish letters are just screwed up and some are just fine. The locale on the server was set up as sv_ISO8859-1.

Here's the real weird thing. When i import this sql file to the mysql database it displays fine on the website.

e.g Väskor/Plånböcker

Now when I click a link on the site that pretty much refreshes the page it becomes

V�skor/Pl�nb�cker->

When I click a link again it turns back to normal (Väskor/Plånböcker), and then click again it goes to this V�skor/Pl�nb�cker->

Sometimes, I can click several links and it's fine and then the next click it changes to V�skor/Pl�nb�cker-> again.

The server locales at the moment are set at UTF-8 and this is the problem I get. If i put the server locale to is8859-1 i get exactly the same issue with different gibberish where ÖÅÄ are supposed to be.

Can anyone tell me what's going on?

 

Do both servers use the same MySQL version?

 

My server is running MySQL 4.1.11 while my old host is running 4.1.20.

I've used apt-get mysql-server-4.1 mysql-common-4.1 to install mine.

I tried to downgrade to 4.0 but it won't run, i get a mysqld.sock error.

My /etc/locale.gen reads

sv_SE UTF-8
sv_SE ISO-8859-1

In phpmyadmin on my server it's set to

MySQL charset: UTF-8 Unicode (utf8)
MySQL connection collation: utf8_general_ci

the database was created like this ...

"create database tdfdbosc character set latin1 collate latin1_swedish_ci;"

On my host, the character set of the tables is reported as latin1.

When I import the sql file into my database through phpmyadmin it looks totally fine. But when i look online i get this switching problem. This is the first time i've ever had a problem with this. I've had character set problems int he past but always managed to fix them by fiddling with the different char set options in the client and host tools.

If it is mysql being a bit older, whats the easiest way to isntall the new one? just downlaod the sources and recompile?

 

i've enclosed two images.

 

I don't think it's a problem with your MySQL versions.
Can you try to export and import the databases with ISO8859-1?

 

sorry to waste your time on this , falko. I found my problem just this second!

The problem was the cache in /tmp. I cleared the directory and the problem disappeared. I've moved the oscommerce cache setting to where i store my sessions and hopefully that will fix.

I was getting really worried the mysql was serving wrong information.

 

hi adam!

i have the same problem:
i migrated a website + myswl-db from a debian 3.1 server to my ubuntu 6.06 server. everything worls fine, but i also get these strange characters.

wow did you solve this problem?
could you drop me some helpful ideas?

bests,

 

download your database to your pc and open the sql file. if you see strange characters in the file you want to change the character encoding that the file is being displayed withe and see if you can "fix" it on the fly. it's usually either utf-8 or iso-8859. I installed kate on my ubuntu system has it's got better options for this, you can simply go into one of its menus and choose which encoding to display. The trick is to know what format your document is in that you exported. Ubuntu is utf-8 by default.

if you use a tool like phpmyadmin or mysql admin you can specifiy the encoding of your document. note, this has nothing to do with the destination encoding on your mysql tables, which is quite often latin1, iso-8859.

once you've imported your sql file correctly, the next thing you want to check is the source file of your html page after rendering.

content="text/html; charset=iso-8859-1">

if you look for a line like this it will tell your browser what to use. you can also toggle this manually in firefox in the view menu and then character encoding. If this is missing it's advisable to put it in so you're browser will sort it out automatically.

 

thanks for yr reply!
unfortunately things haven't changed yet.
might it be a problem with the global charset-settings on my ubuntu-machine?
the thing is, that every website on my machine won't display website correctly.

take a look at the default installation of a phpBB-Forum on my server.
http://www.allocat.net/forum

????

 

hmmm when i load your site it's being detected as UTF-8. When i go to to view->character encoding and select western 8859 in firefox the site displays perfectly.

When i look at your source code for that page , the encoding is not being set (it tells me the site is being viewed as a utf8 site), so it's probably using the encoding of the server which is probably utf8.

your actual data seems fine.

compare my source header with yours

yours

Code:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta http-equiv="Content-Style-Type" content="text/css">

mine

Code:

!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN">
<html dir="LTR" lang="se">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

The difference seems to be that i have lang="se", you probably want lang="de". maybe check the php board settings in the admin panel.

you could try editing /etc/locale.gen and tell the server to use iso-8859-1 instead of utf 8 and see what happens.

 

i changed the code to lang="de" (...) , but this wasn't helpul at all..
i think, i have to change the /etc/locale.gen
the thing is, there is no locale.gen in /etc.

??? ahhh!

 

dpkg-reconfigure locales


does this command work?


otherwise you might want to do

apt-get install locales

then sort it out.

 

yes, but with this results:

Code:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "de_AT:de_DE:de:en_GB:en",
        LC_ALL = (unset),
        LANG = "de_AT"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
Generating locales...
  de_AT.UTF-8... up-to-date
  de_DE.ISO-8859-15@euro... up-to-date
  en_US.UTF-8... up-to-date
Generation complete.
Current default timezone: 'Europe/Vienna'.
Local time is now:      Thu Sep 14 00:09:24 CEST 2006.
Universal Time is now:  Wed Sep 13 22:09:24 UTC 2006.
Run 'tzconfig' if you wish to change it.

 

whats in

/etc/environment

 

Code:

root@lemmi:/etc# more environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games"
LANG="de_AT.UTF-8"
LANGUAGE="de_AT:de_DE:de:en_GB:en"
### BEGIN DEBCONF SECTION FOR localeconf
# Do not edit within this region if you want your changes to be preserved
# by debconf.  Instead, make changes before the "### BEGIN DEBCONF SECTION
# FOR localeconf" line, and/or after the "### END DEBCONF SECTION FOR
# localeconf" line.
LANG=de_AT
### END DEBCONF SECTION FOR localeconf

 

maybe try changing utf-8 to ISO-8859-1?
I must say though, mine is much simpler than yours.

 

hm, not working ...

 

sorry i dont know what else to suggest.

 

but thanks for everything!

 

Run

Code:

apt-get install localeconf

Select locales to install (e.g. en_US ISO-8859-1) and select the standard locale (e.g. en_US).

You will be asked a few questions:

Manage locale configuration files with debconf? <-- Yes
Environment settings that should override the default locale: <-- do not select anything
Replace existing locale configuration files? <-- Yes
Default system locale: <-- e.g. en_US ISO-8859-1