php

Compiling the IMAP extension for PHP 5.3.15 with OS X 10.8.3

I have been testing the Drupal support module locally which features the ability to create tickets from email messages to an IMAP inbox. It requires the imap_open() PHP function provided by the imap PHP extension, which unfortunately is not included in the OS X builds of PHP.

ivucica has published a wonderful script to his blog that compiles the IMAP extension without having to recompile PHP entirely, but unfortunately it was not working for me and nobody else seemed to have my problem either. Compiling the imap library and PCRE went very smoothly, but when it came time to build the PHP extension this error appeared during ./configure:

checking whether build with IMAP works... no
configure: error: build test failed. Please check the config.log for details.

Well, crap. I check config.log and determine it's a linking failure:

configure: program exited with status 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME ""
| #define PACKAGE_TARNAME ""
| #define PACKAGE_VERSION ""
| #define PACKAGE_STRING ""
| #define PACKAGE_BUGREPORT ""
| #define PACKAGE_URL ""
| #define COMPILE_DL_IMAP 1
| #define HAVE_IMAP 1
| #define HAVE_IMAP2000 1
| #define HAVE_IMAP2004 1
| #define HAVE_NEW_MIME2TEXT 1
| #define HAVE_LIBPAM 1
| #define HAVE_IMAP_KRB 1
| #define HAVE_IMAP_SSL 1
| /* end confdefs.h.  */
|
|
| #if defined(__GNUC__) && __GNUC__ >= 4
| # define PHP_IMAP_EXPORT __attribute__ ((visibility("default")))
| #else
| # define PHP_IMAP_EXPORT
| #endif
|
|       PHP_IMAP_EXPORT void mm_log(void){}
|       PHP_IMAP_EXPORT void mm_dlog(void){}
|       PHP_IMAP_EXPORT void mm_flags(void){}
|       PHP_IMAP_EXPORT void mm_fatal(void){}
|       PHP_IMAP_EXPORT void mm_critical(void){}
|       PHP_IMAP_EXPORT void mm_nocritical(void){}
|       PHP_IMAP_EXPORT void mm_notify(void){}
|       PHP_IMAP_EXPORT void mm_login(void){}
|       PHP_IMAP_EXPORT void mm_diskerror(void){}
|       PHP_IMAP_EXPORT void mm_status(void){}
|       PHP_IMAP_EXPORT void mm_lsub(void){}
|       PHP_IMAP_EXPORT void mm_list(void){}
|       PHP_IMAP_EXPORT void mm_exists(void){}
|       PHP_IMAP_EXPORT void mm_searched(void){}
|       PHP_IMAP_EXPORT void mm_expunged(void){}
|       void rfc822_output_address_list(void);
|       void (*f)(void);
|       char foobar () {f = rfc822_output_address_list;}
|
|     char foobar();
|     int main() {
|       foobar();
|       return 0;
|     }
|
configure:6808: result: no
configure:6819: checking whether build with IMAP works
configure:6863: cc -o conftest -g -O2   conftest.c  -Wl,-rpath,/usr/local/imap-2007f/lib -L/usr/local/imap-2007f/lib -lc-client -lpam  -lkrb5  >&5
Undefined symbols for architecture x86_64:
  "_BIO_free", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_BIO_new_mem_buf", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_BIO_new_socket", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_ERR_error_string", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
      _ssl_genkey in libc-client.a(osdep.o)
  "_ERR_get_error", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
      _ssl_genkey in libc-client.a(osdep.o)
  "_ERR_load_crypto_strings", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
  "_EVP_PKEY_free", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_PEM_read_bio_PrivateKey", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_PEM_read_bio_X509", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_RAND_seed", referenced from:
      _ssl_onceonlyinit in libc-client.a(osdep.o)
  "_RSA_generate_key", referenced from:
      _ssl_genkey in libc-client.a(osdep.o)
  "_SSL_CTX_ctrl", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_CTX_free", referenced from:
      _ssl_abort in libc-client.a(osdep.o)
  "_SSL_CTX_load_verify_locations", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_CTX_new", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_CTX_set_cipher_list", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
  "_SSL_CTX_set_default_verify_paths", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_CTX_set_tmp_rsa_callback", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
  "_SSL_CTX_set_verify", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_CTX_use_PrivateKey", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_CTX_use_RSAPrivateKey_file", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
  "_SSL_CTX_use_certificate", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_CTX_use_certificate_chain_file", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
  "_SSL_accept", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
  "_SSL_ctrl", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_free", referenced from:
      _ssl_abort in libc-client.a(osdep.o)
  "_SSL_get_error", referenced from:
      _ssl_getdata in libc-client.a(osdep.o)
      _ssl_sout in libc-client.a(osdep.o)
  "_SSL_get_fd", referenced from:
      _ssl_getdata in libc-client.a(osdep.o)
      _ssl_server_input_wait in libc-client.a(osdep.o)
  "_SSL_get_peer_certificate", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_library_init", referenced from:
      _ssl_onceonlyinit in libc-client.a(osdep.o)
  "_SSL_load_error_strings", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
  "_SSL_new", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_pending", referenced from:
      _ssl_getdata in libc-client.a(osdep.o)
      _ssl_server_input_wait in libc-client.a(osdep.o)
  "_SSL_read", referenced from:
      _ssl_getdata in libc-client.a(osdep.o)
      _ssl_server_input_wait in libc-client.a(osdep.o)
  "_SSL_set_bio", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_set_connect_state", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_set_fd", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
  "_SSL_shutdown", referenced from:
      _ssl_abort in libc-client.a(osdep.o)
  "_SSL_state", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSL_write", referenced from:
      _ssl_start in libc-client.a(osdep.o)
      _ssl_sout in libc-client.a(osdep.o)
  "_SSLv23_client_method", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_SSLv23_server_method", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
  "_TLSv1_client_method", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_TLSv1_server_method", referenced from:
      _ssl_server_init in libc-client.a(osdep.o)
  "_X509_NAME_oneline", referenced from:
      _ssl_open_verify in libc-client.a(osdep.o)
  "_X509_STORE_CTX_get_current_cert", referenced from:
      _ssl_open_verify in libc-client.a(osdep.o)
  "_X509_STORE_CTX_get_error", referenced from:
      _ssl_open_verify in libc-client.a(osdep.o)
  "_X509_free", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_X509_get_ext_d2i", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_X509_get_subject_name", referenced from:
      _ssl_open_verify in libc-client.a(osdep.o)
  "_X509_verify_cert_error_string", referenced from:
      _ssl_open_verify in libc-client.a(osdep.o)
  "_sk_num", referenced from:
      _ssl_start in libc-client.a(osdep.o)
  "_sk_value", referenced from:
      _ssl_start in libc-client.a(osdep.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I couldn't figure out why it wasn't picking up the symbols from libssl, even when manually trying to compile said file and adding a -lssl flag.

After an hour of struggling with it and my debugging efforts going nowhere, I try adding -lcrypto for the hell of it and it works!

tl;dr, if you get this error then simply replace the following line of the aformentioned script:

./configure --with-imap=/usr/local/imap-2007f --with-kerberos --with-imap-ssl

With the following line that adds the required linker flags:

LDFLAGS="-lssl -lcrypto" ./configure --with-imap=/usr/local/imap-2007f --with-kerberos --with-imap-ssl

That's it!

Cryptic MySQL error

Today as I was attempting to test one of my PHP applications, I received this error after attempting to connect to a MySQL database:

Warning:  mysql_connect() [function.mysql-connect]: OK packet 6 bytes shorter
than expected in index.php on line 29

Warning:  mysql_connect() [function.mysql-connect]: mysqlnd cannot connect to
MySQL 4.1+ using old authentication in index.php on line 29

The script giving the error was running on OS X 10.6.4 with the stock PHP 5.3.1. After doing a bit of searching and reading the MySQL documentation on the old password format, I was a bit confused because I ran this on the server:

[user@host ~]# rpm -q mysql mysql-server
mysql-5.0.77-4.el5_5.3
mysql-server-5.0.77-4.el5_5.3

Both the server and client should support the new authentication version, which was introduced all the way back in MySQL 4.1. So why wouldn't it connect?

It turns out that CentOS 5 disables the new password hashes by default in favour of remaining compatible with 3.x (and earlier) MySQL clients. All you have to do is edit /etc/my.cnf and comment the old_passwords=1 line. After restarting the server, you should notice that running SELECT PASSWORD('foobar'); in a MySQL prompt will return 41-character hashes, not the old-style 16 character hashes. Reset the user passwords to start using the new hashes and you'll be good to go.

Some quick PHP performance stats: suPHP vs prefork+mod_php vs itk+mod_php

I have been doing lots of research on how to properly secure PHP on a shared server, especially with regards to finding the best way to sandbox users. On stock apache installations, the apache user must have access to web content in order to serve it which has the unfortunate side effect that every user on the shared hosting server can read the files of every other user.

The solution to them is "sandboxing" them, or in other words having Apache serve each user's web files as that user. I will post a tutorial relatively soon detailing how to do so (along with configuring many other services) but in the mean time here are some benchmarks:

prefork: 2.720166 seconds
suphp: 13.621006 seconds
itk: 4.263002 seconds

These benchmarks were generated using the "ab" benchmark included with the httpd server. They represent the time it took to load the front page of my blog 200 times:
ab -c 1 -n 200 http://www.firewing1.com/
prefork is the standard apache MPM working with mod_php. It's the fastest, but for the reasons outlined above also the most insecure. suPHP tackles the problem by using a SUID executable and running PHP under CGI, but it is extremely slow - even for this modest drupal site, it is just over 5x slower than stock. I compiled the ITK MPM for Apache which also offers the feature of running files under different users but it is based on Prefork and uses mod_php. The performance is still worse (2x slower) than stock, but much better than suPHP.

Tags: 

Installing PHP 5.2.10 on OS X 10.6 Snow Leopard

Since my update to Snow Leopard, I was pleasantly surprised to find that Apple has updated PHP to version 5.3 and also included the GD extension. While I no longer have to rebuild the extension manually like on Leopard, these changes to PHP brought around a different problem: Drupal is currently not compatible with PHP 5.3 (#360605).

I've been trying to get my local Drupal installations working, and although the patch from post #84 works pretty well (when applied to a D6 CVS checkout), Ubercart is still nonfunctional. Since I am currently building and testing Ubercart-enabled sites, my only remaining option was to downgrade to PHP 5.2.10. I wanted to have the same extensions and options that Apple's PHP 5.3 build had, so I started by viewing the output of phpinfo() and copying the configure command. To compile PHP, locally installed copies of libpng, libjpeg and pcre are required so let's started with that:

  1. (Like in the Leopard tutorial, I assume you have installed the Xcode & related developer utilities and that all downloads are saved in the "Downloads" folder in your home). Visit the libpng, libjpeg and PCRE homepages and download the latest release available for both. As of writing, the most recent releases are libjpeg 7, libpng 1.2.39 and PCRE 7.9.
  2. Compile libpng and libjpeg statically:
    cd ~/Downloads && tar xfz libpng-1.2.39.tar.gz
    cd libpng-1.2.39
    ./configure --disable-shared --enable-static
    make && make install DESTDIR=`pwd`/localinstall

    cd ~/Downloads && tar xfz jpegsrc.v7.tar.gz
    cd jpeg-7
    ./configure --disable-shared --enable-static
    make && make install DESTDIR=`pwd`/localinstall

    cd ~/Downloads && tar xfj pcre-7.9.tar.bz2
    cd pcre-7.9
    ./configure --disable-shared --enable-static
    make && make install DESTDIR=`pwd`/localinstall

  3. Since PHP will be built with MySQL support, download and install MySQL x86_64 for OS X. As of writing, the latest version is 5.1.38.
  4. Download PHP 5.2.10, available here
  5. Next, PHP needs to be prepared for compilation. As detailed in PHP bug #49267, a small change is required to get PHP to compile on Snow Leopard:
    1. Type in the terminal:
      cd ~/Downloads && tar xfj php-5.2.10.tar.bz2
      cd php-5.2.10
      nano ext/iconv/iconv.c
    2. Skip down to line 185 (Tip: <ctrl+c> shows current line)
    3. Remove the lib on #define iconv libiconv so that the code reads like this:
      #ifdef HAVE_LIBICONV
      #define iconv iconv
      #endif
    4. Hit <ctrl+o> and to save the file
    5. Hit <ctrl+x> to quit nano
  6. Now, PHP is ready for compilation. We will use a configure command relatively similar to the command extracted from phpinfo() earlier:
    ./configure '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-dependency-tracking' '--sysconfdir=/private/etc' '--with-apxs2=/usr/sbin/apxs' '--enable-cli' '--with-config-file-path=/etc' '--with-libxml-dir=/usr' '--with-openssl=/usr' '--with-kerberos=/usr' '--with-zlib=/usr' '--enable-bcmath' '--with-bz2=/usr' '--enable-calendar' '--with-curl=/usr' '--enable-exif' '--enable-ftp' '--with-gd' '--with-jpeg-dir=/Users/shortname/Downloads/jpeg-7/localinstall/usr/local' '--with-png-dir=/Users/shortname/Downloads/libpng-1.2.39/localinstall/usr/local' '--enable-gd-native-ttf' '--with-ldap=/usr' '--with-ldap-sasl=/usr' '--enable-mbstring' '--enable-mbregex' '--with-mysql=/usr/local/mysql/' '--with-mysqli=/usr/local/mysql/bin/mysql_config' '--with-mysql-sock=/tmp/mysql.sock' '--with-iodbc=/usr' '--enable-shmop' '--with-snmp=/usr' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--with-xmlrpc' '--with-xsl=/usr' '--with-pcre-regex=/Users/shortname/Downloads/pcre-7.9/localinstall/usr/local'

    EXTRA_CFLAGS="-lresolv" make -j2

    Remember to replace shortname in /Users/shortname to your system account's shortname. If you're not sure what that is, type whoami in a terminal to find out.

  7. Finally, backup Snow Leopard's PHP extension so that PHP 5.3 can be restored later, and copy the PHP 5.2.10 extension in its place:
    sudo mv /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.orig106
    sudo cp libs/libphp5.so /usr/libexec/apache2/libphp5.so
  8. The final step is to restart Apache - this can be done by toggling Web Sharing in System Preferences, or alternatively via the apachectl command:
    sudo apachectl restart
  9. That's all! Now run phpinfo() and verify that PHP 5.2.10 is up & running. While I was trying to get this working, I stumbled accross two compile errors - for the sake of completeness, I've listed them below along with the failure cause:

    1. This error occurs if EXTRA_CFLAGS="-lresolv" is not used while compiling PHP:

      Undefined symbols:
        "_res_9_dn_expand", referenced from:
            _zif_dns_get_mx in dns.o
        "_res_9_search", referenced from:
            _zif_dns_get_mx in dns.o
            _zif_dns_check_record in dns.o
        "_res_9_dn_skipname", referenced from:
            _zif_dns_get_mx in dns.o
            _zif_dns_get_mx in dns.o
      ld: symbol(s) not found
      symbols:
        "_res_9_dn_expand", referenced from:
            _zif_dns_get_mx in dns.o
        "_res_9_search", referenced from:
            _zif_dns_get_mx in dns.o
            _zif_dns_check_record in dns.o
        "_res_9_dn_skipname"collect2: , referenced from:
            ld returned 1 exit status_zif_dns_get_mx
      in dns.o
            _zif_dns_get_mx in dns.o
      ld: symbol(s) not found
      collect2: ld returned 1 exit status
    2. This error occurs if the #define iconv libiconv is not changed to #define iconv iconv
      Undefined symbols:
        "_libiconv", referenced from:
            __php_iconv_strlen in iconv.o
            _php_iconv_string in iconv.o
            _php_iconv_string in iconv.o
            __php_iconv_strpos in iconv.o
            __php_iconv_appendl in iconv.o
            __php_iconv_appendl in iconv.o
            _zif_iconv_substr in iconv.o
            _zif_iconv_mime_encode in iconv.o
            _zif_iconv_mime_encode in iconv.o
            _zif_iconv_mime_encode in iconv.o
            _zif_iconv_mime_encode in iconv.o
            _zif_iconv_mime_encode in iconv.o
            _zif_iconv_mime_encode in iconv.o
            _php_iconv_stream_filter_append_bucket in iconv.o
            _php_iconv_stream_filter_append_bucket in iconv.o
      ld: symbol(s) not found
      collect2: ld returned 1 exit status

Adding support for the GD graphic library on OS X 10.5 Leopard

I play around with a number of Drupal locally and today I finally caved in and decided to add GD support to OS X's webserver. Although I've heard great things about the Entropy PHP packages, I did not want to replace OS X's integrated php installation so my remaining option was to grab a copy of PHP 5.2.8 (the version bundled with Leopard) and compile the module myself. I know there must be quite a few other people who are looking do to the same thing, so I here's a step-by-step guide on how I did it:

(Please note that I assume you have Xcode installed, and that all downloads are saved in the "Downloads" folder in your home. If this is not the case, please move the downloads there first before executing any of the listed commands)

  1. Visit the libpng and libjpeg homepages and download the latest release available for both. As of writing, the most recent releases are libjpeg 7 and libpng 1.2.37 - the commands below will need to be adjusted slightly if you are compiling a more recent version.
  2. Download PHP 5.2.8, available here
  3. Extract PHP and prepare to compile libpng and libjpeg statically:
    cd ~/Downloads
    tar xfj php-5.2.8.tar.bz2
    cd php-5.2.8/ext/gd
    mv ~/Downloads/libpng-1.2.37.tar.gz ~/Downloads/jpegsrc.v7.tar.gz .
  4. Compile libpng without shared libraries:
    tar xfz libpng-1.2.37.tar.gz && cd libpng-1.2.37
    ./configure --disable-shared --enable-static
    make && make install DESTDIR=`pwd`/localinstall
    cd ../
  5. Compile libjpeg without shared libraries:
    tar xfz jpegsrc.v7.tar.gz && cd jpeg-7
    cp /usr/share/libtool/config.* .
    ./configure --disable-shared --enable-static
    make && make install DESTDIR=`pwd`/localinstall
    cd ../
  6. Compile the GD extension and point to our newly compiled libpng and libjpeg static libraries:
    cd ~/Downloads/php-5.2.8/ext/gd
    phpize
    MACOSX_DEPLOYMENT_TARGET=10.5 \
    CFLAGS="-O3 -fno-common -arch i686" \
    LDFLAGS="-O3 -arch i686" \
    CXXFLAGS="-O3 -fno-common -arch i686" \
    ./configure --with-zlib-dir=/usr \
    --with-png-dir=`pwd`/libpng-1.2.37/localinstall/usr/local \
    --with-jpeg-dir=`pwd`/jpeg-7/localinstall/usr/local \
    --with-freetype-dir=/usr/X11R6 \
    --with-xpm-dir=/usr/X11R6
    make
    sudo make install
  7. Copy /private/etc/php.ini.default to /private/etc/php.ini
    • Change the default 'extension_dir' (./) to /usr/lib/php/extensions/no-debug-non-zts-20060613/
    • Add extension=gd.so a bit below extension_dir to enable the gd extension
  8. sudo nano /System/Library/LaunchDaemons/org.apache.httpd.plist
    Change the part between <array> and </array> so that it looks like this:

    <array>
    <string>/usr/bin/arch</string>
    <string>-i386</string>
            <string>/usr/sbin/httpd</string>
            <string>-D</string>
            <string>FOREGROUND</string>
    </array>

    This will ensure that Apache starts in 32 bit mode, which is needed as the gd extension was compiled for i686.

  9. That's all! After a reboot (or toggle Web sharing in the System Preferences), everything should be working. phpinfo() should now confirm that GD library is indeed loaded. If it isn't or if apache refuses to start, try opening Console to check /var/log/apache2/error_log for some useful pointers as to what's wrong.

    Helpful source I used for writing this tutorial: HOWTO: Install Habari on Mac OS X Leopard, From Scratch