Porting to OS/2: Case #1 line ending
When porting programs and libraries from Unix to OS/2, one of the most problematic cases is a line ending problem.
OS/2 including Windows and DOS uses CR/LF pair to mark an end of line. Whereas, Unix-like systems use LF only. And OS/2 distinguish text mode(aka translation mode) from binary mode(aka no translation mode). That is, OS/2 translates LF to CR/LF when writing to devices, and CR/LF to LF when reading from on text mode. Whereas, Unix-like systems don't care about distinguishing text mode from binary mode. In OS/2's terms, Unix-like systems treat all the devices as binary mode. And the default mode of OS/2 is text mode. These are the causes of a line ending problem.
For example, if we open a file with open() without both O_TEXT and O_BINARY, it is opened in text mode. Likewise, fopen() without both "t" and "b", it is opened in text mode. But on Unix-like systems all of them are opened in binary mode.
If they are real text files, nothing is problematic. But if they are binary files, then OS/2 ports get corrupted datum, as a result, unexpected behavior occurs.
To overcome line ending problem, we should look into all open() and fopen(), know what they do. If they manipulate binary files, then we have to add O_BINARY and "b" to open() and fopen(), respectively. This is not an easy work. But there are some hints. As I said before, Windows also uses the same EOL style. And many projects were already ported to Windows. So it makes your life easier to find Windows specific codes.
Usually, Windows specific codes are guarded by WIN32, _WIN32, _MSC_VER, __MINGW32__ or __CYGWIN__ macros. So find them by grep like this.
Then, you can get a list of the files containg MINGW. If the codes are useful for OS/2, then add OS/2 macros to guard OS/2 codes.
On OS/2 kLIBC, gcc provides a few pre-defined macros. They are __OS2__, __EMX__, __KLIBC__, and __INNOTEK_LIBC__. I recommend to use __OS2__ if possible. And if it is specific to kLIBC, then __KLIBC__. Because it is shorter than __INNOTEK_LIBC__. ^^
Then, how to guard with those macros ? Use #ifdef or #if with defined().
For examples, consider Windows codes like this.
Then modify it like this.
Or if there are some codes that Windows does not want, then
And OS/2 does not like it, too. Then,
Note that [#ifdef macro] is equivalent to [#if defined(macro)] and [#ifndef macro] to [#if !defined(macro)].
And more easy way is to find O_BINARY. So grepping O_BINARY makes your life much easier. If you find O_BINARY, then do like the above to guard OS/2 codes. Although many projects uses open(), there are also projects using fopen() as well. So do not miss to check fopen(), too.
Finally, kLIBC provides a special way to do this more easily. It is -Zbin-files linker flag. If you pass -Zbin-files to compiler when linking, the default translation mode becomes binary mode like Unix-like systems. You can pass -Zbin-files to compiler with LDFLAGS. When you configure, set LDFLAGS. For examples,
Or you can use GCCOPT env var. gcc treats GCCOPT as arguments. So this is useful when checking whether or not some flags are needed before configuring really. If you have a doubt that some missing flags are the causes of mis-behavior, you can add those flags to GCCOPT. In this case, you can set GCCOPT like this before linking.
As I said before, -Zbin-files is a linker flag, so compilation is regardless of this flag. And remember that it would be better to use GCCOPT just only in order to test flags except some default arguments such as -pipe and -static-libgcc.
Another way is to modify configure.ac or configure.in. I recommend this, but this requires additional and complicated works. But with this, you can incorpoate your patch to the upstream. In addition, you can add more complicated feature test to configure. And you don't have to distirbute extra files except original sources.
Anyway, let's see how to do this.
First, find pre-existed OS-specific codes. Then add OS/2 codes to the codes following the convention. For examples, libmad v0.15.1b has the following codes.
Then, add OS/2 codes after mingw codes like this,
For references, use [*-*-os2*] for $host and [os2*] for $host_os.
Or, you can use AS_CASE().
If this flag is needed for configure test, then these codes should be placed before testing codes of configure, else it's ok if before AC_OUTPUT().
By the way, If you modified configure.ac or configure.in, then you have to have the same version of autotools as the original distribution to build. If not, you have to regenerate build system with your autotools. Just do autoreconf.
There is the way equivalent to this. It is to use _fmode_bin global variable like this.
This doesn't require any additional flags. But if there are many executable, then you should add this codes to all main() of those executable. And maybe for DLLs, too.
However, -Zbin-files also does not change the mode of stdio handle such as stdin, stdout and stderr. For them, use setmode().
isatty() was used to prevent from setting consol to binary mode. Otherwise, you would see stairs on your screen. ^^
However, be careful. OS/2 native text files have CR/LF not LF. So manipulating OS/2 text files with this setting, may cause unexpected behavior like manipulating binary files on text mode. Again, use -Zbin-files carefully.
One more thing, remember that pipe() opens a pipe in text mode by default. Of course, in binary mode with -Zbin-files. If needed, do setmode() for pipes as well.
So far, we looked into the way to overcome line ending problem. If you solve this, you will go forward much.
// ----- 2017/09/24
Some functions such as mkstemp() and pipe() open a file in text mode. Do not forget them. Use setmode() to switch the file handles to binary mode.
// -----
OS/2 including Windows and DOS uses CR/LF pair to mark an end of line. Whereas, Unix-like systems use LF only. And OS/2 distinguish text mode(aka translation mode) from binary mode(aka no translation mode). That is, OS/2 translates LF to CR/LF when writing to devices, and CR/LF to LF when reading from on text mode. Whereas, Unix-like systems don't care about distinguishing text mode from binary mode. In OS/2's terms, Unix-like systems treat all the devices as binary mode. And the default mode of OS/2 is text mode. These are the causes of a line ending problem.
For example, if we open a file with open() without both O_TEXT and O_BINARY, it is opened in text mode. Likewise, fopen() without both "t" and "b", it is opened in text mode. But on Unix-like systems all of them are opened in binary mode.
If they are real text files, nothing is problematic. But if they are binary files, then OS/2 ports get corrupted datum, as a result, unexpected behavior occurs.
To overcome line ending problem, we should look into all open() and fopen(), know what they do. If they manipulate binary files, then we have to add O_BINARY and "b" to open() and fopen(), respectively. This is not an easy work. But there are some hints. As I said before, Windows also uses the same EOL style. And many projects were already ported to Windows. So it makes your life easier to find Windows specific codes.
Usually, Windows specific codes are guarded by WIN32, _WIN32, _MSC_VER, __MINGW32__ or __CYGWIN__ macros. So find them by grep like this.
- grep -ri MINGW *
Then, you can get a list of the files containg MINGW. If the codes are useful for OS/2, then add OS/2 macros to guard OS/2 codes.
On OS/2 kLIBC, gcc provides a few pre-defined macros. They are __OS2__, __EMX__, __KLIBC__, and __INNOTEK_LIBC__. I recommend to use __OS2__ if possible. And if it is specific to kLIBC, then __KLIBC__. Because it is shorter than __INNOTEK_LIBC__. ^^
Then, how to guard with those macros ? Use #ifdef or #if with defined().
For examples, consider Windows codes like this.
| 1 2 3 | #ifdef __MINGW32__ /* some codes */ #endif |
Then modify it like this.
| 1 2 3 | #if defined(__MINGW32__) || defined(__OS2__) /* some codes */ #endif |
Or if there are some codes that Windows does not want, then
| 1 2 3 | #ifndef __MINGW32__ /* some codes */ #endif |
And OS/2 does not like it, too. Then,
| 1 2 3 | #if !defined(__MINGW32__) && !defined(__OS2__) /* some codes */ #endif |
Note that [#ifdef macro] is equivalent to [#if defined(macro)] and [#ifndef macro] to [#if !defined(macro)].
And more easy way is to find O_BINARY. So grepping O_BINARY makes your life much easier. If you find O_BINARY, then do like the above to guard OS/2 codes. Although many projects uses open(), there are also projects using fopen() as well. So do not miss to check fopen(), too.
Finally, kLIBC provides a special way to do this more easily. It is -Zbin-files linker flag. If you pass -Zbin-files to compiler when linking, the default translation mode becomes binary mode like Unix-like systems. You can pass -Zbin-files to compiler with LDFLAGS. When you configure, set LDFLAGS. For examples,
| 1 2 3 4 5 | extproc sh export LDFLAGS="-Zbin-files" ./configure |
Or you can use GCCOPT env var. gcc treats GCCOPT as arguments. So this is useful when checking whether or not some flags are needed before configuring really. If you have a doubt that some missing flags are the causes of mis-behavior, you can add those flags to GCCOPT. In this case, you can set GCCOPT like this before linking.
- set GCCOPT=%GCCOPT% -Zbin-files
As I said before, -Zbin-files is a linker flag, so compilation is regardless of this flag. And remember that it would be better to use GCCOPT just only in order to test flags except some default arguments such as -pipe and -static-libgcc.
Another way is to modify configure.ac or configure.in. I recommend this, but this requires additional and complicated works. But with this, you can incorpoate your patch to the upstream. In addition, you can add more complicated feature test to configure. And you don't have to distirbute extra files except original sources.
Anyway, let's see how to do this.
First, find pre-existed OS-specific codes. Then add OS/2 codes to the codes following the convention. For examples, libmad v0.15.1b has the following codes.
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | if test "$GCC" = yes then case "$host" in *-*-mingw*) case "$build" in *-*-cygwin*) CPPFLAGS="$CPPFLAGS -mno-cygwin" LDFLAGS="$LDFLAGS -mno-cygwin" ;; esac ;; esac dnl case "$host" in dnl *-*-cygwin* | *-*-mingw*) dnl LDFLAGS="$LDFLAGS -no-undefined -mdll" dnl ;; dnl esac fi |
Then, add OS/2 codes after mingw codes like this,
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | if test "$GCC" = yes then case "$host" in *-*-mingw*) case "$build" in *-*-cygwin*) CPPFLAGS="$CPPFLAGS -mno-cygwin" LDFLAGS="$LDFLAGS -mno-cygwin" ;; esac ;; *-*-os2*) LDFLAGS="$LDFLAGS -Zbin-files" ;; esac dnl case "$host" in dnl *-*-cygwin* | *-*-mingw*) dnl LDFLAGS="$LDFLAGS -no-undefined -mdll" dnl ;; dnl esac fi |
For references, use [*-*-os2*] for $host and [os2*] for $host_os.
Or, you can use AS_CASE().
| 1 2 | AS_CASE([$host_os], [os2*], [LDFLAGS="$LDFLAGS -Zbin-files"]) |
If this flag is needed for configure test, then these codes should be placed before testing codes of configure, else it's ok if before AC_OUTPUT().
By the way, If you modified configure.ac or configure.in, then you have to have the same version of autotools as the original distribution to build. If not, you have to regenerate build system with your autotools. Just do autoreconf.
- sh autoreconf -fvi
There is the way equivalent to this. It is to use _fmode_bin global variable like this.
| 1 2 3 4 5 6 7 8 9 10 11 12 | int main( void ) { #ifdef __OS2__ extern int _fmode_bin; _fmode_bin = 1; #endif /* some codes */ return 0; } |
This doesn't require any additional flags. But if there are many executable, then you should add this codes to all main() of those executable. And maybe for DLLs, too.
However, -Zbin-files also does not change the mode of stdio handle such as stdin, stdout and stderr. For them, use setmode().
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | #include <io.h> #include <fcntl.h> int main( void ) { #ifdef __OS2__ if( !isatty( fileno( stdin ))) setmode( fileno( stdin ), O_BINARY ); if( !isatty( fileno( stdout ))) setmode( fileno( stdout ), O_BINARY ); if( !isatty( fileno( stderr ))) setmode( fileno( stderr ), O_BINARY ); #endif /* some codes */ return 0; } |
isatty() was used to prevent from setting consol to binary mode. Otherwise, you would see stairs on your screen. ^^
However, be careful. OS/2 native text files have CR/LF not LF. So manipulating OS/2 text files with this setting, may cause unexpected behavior like manipulating binary files on text mode. Again, use -Zbin-files carefully.
One more thing, remember that pipe() opens a pipe in text mode by default. Of course, in binary mode with -Zbin-files. If needed, do setmode() for pipes as well.
So far, we looked into the way to overcome line ending problem. If you solve this, you will go forward much.
// ----- 2017/09/24
Some functions such as mkstemp() and pipe() open a file in text mode. Do not forget them. Use setmode() to switch the file handles to binary mode.
// -----
댓글
댓글 쓰기