Perl CGI picture-of-the-day script not behaving as expected after migration to new server – Apache

I have a perl cgi script that has been working perfectly for me on a FreeBSD 5.4 Apache 1.3 webserver for many years without trouble. It’s a picture of the day script that randomly selects a picture from a given directory for inclusion on an shtml page with the server side include

<!--#exec cgi="/cgi-bin/pod/pod.cgi"-->

I recently migrated to a new server on Google Cloud Platform – Debian 9 (Stretch), Apache 2.4. And the script broke. After setting the server configurations to execute cgi perl scripts correctly, and re-uploading the script in ASCII, the script began working again, but now with an anomalous behavior. Instead of displaying one image (the same image) all day long and then changing the image at midnight (the desired behavior) it is now changing the image every time the page is reloaded in a web browser.

The script uses a flat-file log that keeps track of which images have been used from the source directory, and doesn’t repeat displaying of any images until all images from the target directory have been used (logged in pod.log). When working correctly, it will display a new image every day (changing at midnight), which will remain the same for all users, whether the page is reloaded or not, until the following midnight.

Permissions have all been set on the necessary files as specified in the comments of the script. The script has been uploaded to the server in ASCII format (will not work at all if uploaded in binary). The script is displaying an image from the correct directory. BUT….. every time the page is refreshed, a new images is loaded and logged to the pod.log file.

One thing that I thought might be affecting the script was where it was getting the time for the date/time function of the script. When I entered the “date” command from the debian command prompt the server returned the correct time that I had configured the server to – America/Los_Angeles. But I noticed that when files on my webserver were touched or changed, it was time-stamping them with UTC time, which is 8 hours later. Thinking that Apache might be causing the different time-stamp, I tried changing the time-zone in php.ini for apache2. This didn’t seem to change anything (after apache2ctl restart), so I thought, maybe I’ll change the server timezone to UTC. If you can’t beat ’em, join ’em. Right? Well that made it so the “date” command from the debian command line returned the time in UTC. Also noted: files on the webserver were still time-stamping with UTC time zone. All was looking good! But then I checked the time that was being used by perl/cgi with this little gem, which returns the date and time in a human-readable format…

#!/usr/bin/perl
print "Content-type: text/htmlnn";
@months = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec);
@weekDays = qw(Sun Mon Tue Wed Thu Fri Sat Sun);
($second, $minute, $hour, $dayOfMonth, $month, $yearOffset, $dayOfWeek, $dayOfYear, $daylightSavings) = localtime();
$year = 1900 + $yearOffset;
$theTime = "$hour:$minute:$second, $weekDays[$dayOfWeek] $months[$month] $dayOfMonth, $year";
print $theTime; 

…And this script was returning the time in the time zone America/Los_Angeles, rather than UTC. Now I don’t know if this discrepancy could be the thing causing the bug in my pic of the day script. But my guess, with my very limited experience is telling me that it’s at least possible. But I’ve taken this debugging/troubleshooting of the script about as far as my technical abilities will take me.

I need to know:

1. what is causing the script to return a new picture and update the log file on every reload, rather than remaining static for 1 day?
2. is it caused by an unintentional (fat finger) error in the script?
3. is it caused by some different way my new server and/or its configuration are handling the script?
4. is it caused by the time-stamp/time-zone issues I mentioned in the previous paragraph?
5. or is this a result of something I’m completely missing?

Next I’ll provide the source code for the script and my apache configuration files. EXAMPLE.COM should be replaced by your domain name wherever it appears and paths to files should be adjusted to your locations.

#!/usr/bin/perl

##############################################################
#  POD (Picture of the Day) Version 1.30
##############################################################

package VAR;
use strict;

##############################################################
#  Installation
##############################################################

#  1. Edit path to Perl at top of script (pod.cgi) if it
#  differs on your server. Usual and default path it
#  [/usr/bin/perl]. However, on some servers it may be
#  /usr/local/bin/perl. If in doubt, then refer to a script on
#  your server that does work, or ask your web host. Edit
#  variables below. Ensure you edit (and save) the script using
#  an ASCII editor like Notepad.
#
#  2. Via FTP, create directory on server in CGI-BIN called
#  pod. No need to CHMOD - you can leave set to server
#  default directory permissions.
#
#  3. Via FTP, create subdirectory in 'pod' directory
#  called data and CHMOD 777 (drwxrwxrwx).
#
#  4. FTP upload the pod.cgi script to the 'pod'
#  directory in ASCII (text) and CHMOD 755 (rwxr-xr-x). You may
#  need to rename the scripts with the .pl extension if your
#  server uses the .pl extension for CGI-Perl scripts.
#
#  images/      755 (drwxr-xr-x)
#  cgi-bin/pod/
#   pod.cgi     755 (rwxr-xr-x)
#   data/       777 (drwxrwxrwx)
#

##############################################################
#  Operation
##############################################################
#
#  METHOD 1: SSI Method
#  ====================
#  Call the script via SSI (Server-Side Includes). The image
#  is embedded in the page. Insert the following SSI tag in
#  the desired page:
#
#  <!--#exec cgi="/cgi-bin/pod/pod.cgi"-->
#
#  In either case, ensure to replace the cgi-bin/pod/ portion
#  of the SSI tag with your path to the script.
#
#  If you get the [an error occurred while processing this
#  directive] error message or no image / message displays,
#  make sure (a) the path to Perl is correct, (b) the script
#  was uploaded in ASCII, (c) the script is chmod 755
#  (rwxr-xr-x) and (d) the path to the script in the SSI tag
#  is correct - if in doubt, then ask your web host. If still
#  problematic then try the following:
#
#  1. On most servers, the page with a SSI tag must be named
#  with the SHTML extension in order for the server to parse
#  and execute the SSI tag. Check the page source. If you
#  still see the SSI tag, then it was not parsed. Try
#  renaming the page with the SHTML extension. If the SSI tag
#  is still not parsed (and still visible), then SSI may not
#  be enabled on your server - ask your web host.
#
#  2. Try calling the script directly from the browser. If
#  you get a server 500 error, then check your server error
#  logs.
#
#  3. You can also try the following SSI tag:
#
#  <!--#include virtual="/cgi-bin/pod/pod.cgi"-->
#
#  METHOD 1: Non-SSI Method
#  ====================
#  You can also call the script directly from the browser:
#
#  http://www.yourdomain.com/cgi-bin/pod/pod.cgi
#
#  The image is NOT embedded, but is instead displayed in a
#  script generated HTML page.

##############################################################
#  Configuration
##############################################################

#  Full (absolute) server directory path of directory holding
#  image files for the POD script to draw from. Create this
#  directory in advance and upload images (in Binary) to this
#  directory. No need to chmod. NO trailing slash at end of
#  path.
$VAR::image_dir = "/var/www/EXAMPLE.COM/httpdocs/pod";

#  URL of directory holding image files for the POD script to
#  draw from. NO trailing slash at end of URL.
$VAR::image_url = "http://www.EXAMPLE.COM/pod";

#  Full (absolute) server directory path for script data files
#  (pod.log, pod.err). Create this directory in advance and
#  chmod (777 or drwxrwxrwx). NO trailing slash at end of path.
$VAR::data_dir  = "/var/www/EXAMPLE.COM/httpdocs/pod/data";

#  Output template - how POD image (or error message) is
#  displayed. Feel free to change the HTML but (1) the MS link
#  back MUST be retained and (2) the <%image%> tag MUST be
#  retained as the tag is replaced with the image (or error
#  message) HTML code.
$VAR::template  = qq~
<center>
<table border="1">
        <th>
        <%image%>
        </th>
</table>
</center>
~;

##########################################################################
#  Do NOT change or alter the code below!
##########################################################################

eval {
    ($0 =~ m,(.*)/[^/]+,) && unshift (@INC, "$1");
    require 5.004;
};
if ($@) {
    print "Content-type: text/htmlnn";
    print "Server Error Message: $@n";
    exit;
}

eval { &main; };
if ($@) { &error ("[Error 01]: $@"); }
exit;

###############################################
# Main
###############################################
sub main {

my ($time, $date) = &get_time_stamp();
my $num;

if (-e "$VAR::data_dir/pod.log") {
    open (LOG, "$VAR::data_dir/pod.log") ||
    &error ("Error [02]: Cannot open pod.log file - $!");
    my @entries = <LOG>;
    close (LOG);
    chomp (@entries);

    my @match = grep (/^$date/, @entries);
    if (@match) {
        foreach (@match) {
            split (/|/);
            if ($_[0] eq $date) {
                $num = $_[1];
                last;
            }
        }
    }
}

opendir (DIR, "$VAR::image_dir") || &error ("Error [03]: Cannot open $VAR::image - $!");
my @files = sort (grep { m/.*.gif|.jpg/ } readdir (DIR));
closedir (DIR);

if ($num eq "") { $num = int (rand @files); }
my $image = @files[$num];
if (! -e "$VAR::image_dir/$image") { &error ("Error [04]: Cannot find image file [$image]"); }

my $tag = "<img src="$VAR::image_url/$image">";
$VAR::template =~ s/<%image%>/$tag/gis;
print $VAR::template;

my ($found, $newfile);
if (-e "$VAR::data_dir/pod.log") {
    open (LOG, "$VAR::data_dir/pod.log") ||
    &error ("Error [05]: Cannot open pod.log file - $!");
    my @entries = <LOG>;
    close (LOG);
    chomp (@entries);

    foreach (@entries) {
        split (/|/);
        if ($_[0] eq $date) {
            $_[2]++;
            $newfile .= "$date|$_[1]|$_[2]|$_[3]n";
            $found++;
        }       
        else { $newfile .= "$_n"; }
    }
    if (! $found) { $newfile .= "$date|$num|1|$imagen"; }

    open (LOG, ">$VAR::data_dir/pod.log") ||
    &error ("Error [06]: Cannot open pod.log file - $!");
    flock (LOG, 2) || &error ("Error [07]: Cannot lock pod.log file - $!");
    print LOG $newfile;
    close (LOG);
}
else {
    open (LOG, ">$VAR::data_dir/pod.log") ||
    &error ("Error [08]: Cannot open pod.log file - $!");
    print LOG "$date|$num|1|$imagen";
    close (LOG);
    chmod (0666, "$VAR::data_dir/pod.log") ||
    &error ("Error [09]: Cannot chmod pod.log file - $!");
}
}

###############################################
# Get Time Stamp
###############################################
sub get_time_stamp {

my (@tb)    = localtime (time);
my ($ap)    = "am";

$tb[4]++;
for (0..4) { $tb[$_] = sprintf ("%02d", $tb[$_]); }
$tb[5] += 1900;
$ap = "pm" if ($tb[2] >= 12);
$tb[2] -= 12 if ($tb[2] > 12);

my $date = "$tb[4]/$tb[3]/$tb[5]";
return ("$tb[2]:$tb[1]:$tb[0]$ap $date", $date);
}

###############################################
# Error Handler
###############################################
sub error {

my $error       = shift;
my ($time, $date)   = &get_time_stamp();

my $tag         = "Cannot display image";
$VAR::template      =~ s/<%image%>/$tag/gis;

print $VAR::template;

open (ERR, ">>$VAR::data_dir/pod.err");
print ERR "$time | $ENV{'REMOTE_ADDR'} | $errorn";
close (ERR);
chmod (0666, "$VAR::data_dir/pod.err");
exit;
}
########################################
#end of Picture of the Day script
########################################

This is my apache2.conf (again, I’ve changed my domain name to EXAMPLE.COM wherever my domain name appears…

# configuration directives that give the server its instructions.
# See http://httpd.apache.org/docs/2.4/ for detailed information about
# the directives and /usr/share/doc/apache2/README.Debian about Debian specific
# hints.
#
#
# Summary of how the Apache 2 configuration works in Debian:
# The Apache 2 web server configuration in Debian is quite different to
# upstream's suggested way to configure the web server. This is because Debian's
# default Apache2 installation attempts to make adding and removing modules,
# virtual hosts, and extra configuration directives as flexible as possible, in
# order to make automating the changes and administering the server as easy as
# possible.

# It is split into several files forming the configuration hierarchy outlined
# below, all located in the /etc/apache2/ directory:
#
#       /etc/apache2/
#       |-- apache2.conf
#       |       `--  ports.conf
#       |-- mods-enabled
#       |       |-- *.load
#       |       `-- *.conf
#       |-- conf-enabled
#       |       `-- *.conf
#       `-- sites-enabled
#               `-- *.conf
#
#
# * apache2.conf is the main configuration file (this file). It puts the pieces
#   together by including all remaining configuration files when starting up the
#   web server.
#
# * ports.conf is always included from the main configuration file. It is
#   supposed to determine listening ports for incoming connections which can be
#   customized anytime.
#
# * Configuration files in the mods-enabled/, conf-enabled/ and sites-enabled/
#   directories contain particular configuration snippets which manage modules,
#   global configuration fragments, or virtual host configurations,
#   respectively.
#
#   They are activated by symlinking available configuration files from their
#   respective *-available/ counterparts. These should be managed by using our
#   helpers a2enmod/a2dismod, a2ensite/a2dissite and a2enconf/a2disconf. See
#   their respective man pages for detailed information.
#
# * The binary is called apache2. Due to the use of environment variables, in
#   the default configuration, apache2 needs to be started/stopped with
#   /etc/init.d/apache2 or apache2ctl. Calling /usr/bin/apache2 directly will not
#   work with the default configuration.


# Global configuration
#

#
# ServerRoot: The top of the directory tree under which the server's
# configuration, error, and log files are kept.
#
# NOTE!  If you intend to place this on an NFS (or otherwise network)
# mounted filesystem then please read the Mutex documentation (available
# at <URL:http://httpd.apache.org/docs/2.4/mod/core.html#mutex>);
# you will save yourself a lot of trouble.
#
# Do NOT add a slash at the end of the directory path.
#
#ServerRoot "/etc/apache2"

# Set timezone for apache
  SetEnv TZ America/Los_Angeles

#
# The accept serialization lock file MUST BE STORED ON A LOCAL DISK.
#
Mutex file:${APACHE_LOCK_DIR} default

#
# The directory where shm and other runtime files will be stored.
#

DefaultRuntimeDir ${APACHE_RUN_DIR}

#
# PidFile: The file in which the server should record its process
# identification number when it starts.
# This needs to be set in /etc/apache2/envvars
#
PidFile ${APACHE_PID_FILE}

#
# Timeout: The number of seconds before receives and sends time out.
#
Timeout 300

#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On

#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 500

#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 5


# These need to be set in /etc/apache2/envvars
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}

#
# HostnameLookups: Log the names of clients or just their IP addresses
# e.g., www.apache.org (on) or 204.62.129.132 (off).
# The default is off because it'd be overall better for the net if people
# had to knowingly turn this feature on, since enabling it means that
# each client request will result in AT LEAST one lookup request to the
# nameserver.
#
HostnameLookups Off

# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here.  If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog ${APACHE_LOG_DIR}/error.log

#
# LogLevel: Control the severity of messages logged to the error_log.
# Available values: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the log level for particular modules, e.g.
# "LogLevel info ssl:warn"
#
LogLevel warn

# Include module configuration:
IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf

# Include list of ports to listen on
Include ports.conf


# Sets the default security model of the Apache2 HTTPD server. It does
# not allow access to the root filesystem outside of /usr/share and /var/www.
# The former is used by web applications packaged in Debian,
# the latter may be used for local directories served by the web server. If
# your system is serving content from a sub-directory in /srv you must allow
# access here, or in any related virtual host.
<Directory />
        Options FollowSymLinks
        AllowOverride None
        Require all denied
</Directory>

<Directory /usr/share>
        AllowOverride None
        Require all granted
</Directory>

<Directory /var/www/>
        Options -Indexes +FollowSymLinks
        AllowOverride All
        Require all granted
</Directory>

#<Directory /srv/>
#       Options Indexes FollowSymLinks
#       AllowOverride None
#       Require all granted
#</Directory>
# AccessFileName: The name of the file to look for in each directory
# for additional configuration directives.  See also the AllowOverride
# directive.
#
AccessFileName .htaccess
#
# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
#
<FilesMatch "^.ht">
        Require all denied
</FilesMatch>
#
# The following directives define some format nicknames for use with
# a CustomLog directive.
#
# These deviate from the Common Log Format definitions in that they use %O
# (the actual bytes sent including headers) instead of %b (the size of the
# requested file), because the latter makes it impossible to detect partial
# requests.
#
# Note that the use of %{X-Forwarded-For}i instead of %h is not recommended.
# Use mod_remoteip instead.
#
LogFormat "%v:%p %h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" vhost_combined
LogFormat "%h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" combined
LogFormat "%h %l %u %t "%r" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
# Include of directories ignores editors' and dpkg's backup files,
# see README.Debian for details.
# Include generic snippets of statements
IncludeOptional conf-enabled/*.conf
# Include the virtual host configurations:
IncludeOptional sites-enabled/*.conf
# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

And this is the conf for my virtual host…

<VirtualHost *:80>
    ServerName EXAMPLE.com
    ServerAlias www.EXAMPLE.com
    UseCanonicalName Off
    ServerAlias  EXAMPLE1.com
    ServerAlias  www.EXAMPLE1.com
    ServerAlias  EXAMPLE2.com
    ServerAlias  www.EXAMPLE2.com
    ServerAlias  EXAMPLE.co.uk
    ServerAlias  www.EXAMPLE.co.uk
    ServerAlias  EXAMPLE.net
    ServerAlias  www.EXAMPLE.net
    ServerAlias  EXAMPLE3.com
    ServerAlias  www.EXAMPLE3.com
    ServerAdmin [email protected]
    DocumentRoot /var/www/EXAMPLE.com/httpdocs
    <Directory /var/www/EXAMPLE.com/httpdocs>
        Options -Indexes +FollowSymLinks
        AllowOverride All
    </Directory>
    ScriptAlias "/cgi-bin/" "/var/www/EXAMPLE.com/cgi-bin/"
    #<Directory "/var/www/EXAMPLE.com/cgi-bin/">
    #    Options +ExecCGI
    #    AddHandler cgi-script .cgi
    #    AllowOverride All    
    #</Directory>
    #<Directory "/var/www/EXAMPLE.com/httpdocs/members/cgi-bin">
    #    Options +ExecCGI
    #    AddHandler cgi-script .cgi
    #    AllowOverride All
    #</Directory>
    #<Directory "/var/www/EXAMPLE.com/httpdocs/pod">
    #    Options +ExecCGI
    #    AddHandler cgi-script .cgi
    #    AllowOverride All
    #</Directory>

    Alias "/passwd/" "/var/www/EXAMPLE.com/passwd/"

    <IfModule mod_ssl.c>
        SSLEngine off
    </IfModule>

    <Directory /var/www/EXAMPLE.com>
        Options +ExecCGI +FollowSymLinks +Includes
        AddHandler cgi-script .cgi
        AllowOverride All
    </Directory>

    <Directory /var/www/EXAMPLE.com>
        <IfModule sapi_apache2.c>
                php_admin_flag engine on
                php_admin_flag safe_mode on
                php_admin_value open_basedir "/var/www/EXAMPLE.com/httpdocs:/tmp"
        </IfModule>
        <IfModule mod_php5.c>
                php_admin_flag engine on
                php_admin_flag safe_mode on
                php_admin_value open_basedir "/var/www/EXAMPLE.com/httpdocs:/tmp"
        </IfModule>
    </Directory>

    <Directory /var/www/EXAMPLE.com>
        RewriteEngine on

        # the following section prevents outside sites from hot-linking photos
        # leave this next line in allow empty referrers, remove to disallow empty referrers
        RewriteCond %{HTTP_REFERER} !^$ [NC]
        RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+.)*XX.XXX.XXX.XXX(:[0-9]+)?(/.*)?$ [NC]
        RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+.)*EXAMPLE.com(:[0-9]+)?(/.*)?$ [NC]
        RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+.)*EXAMPLE.org(:[0-9]+)?(/.*)?$ [NC]
        RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+.)*EXAMPLE.net(:[0-9]+)?(/.*)?$ [NC]
        RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+.)*EXAMPLE.co.uk(:[0-9]+)?(/.*)?$ [NC]
        RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+.)*EXAMPLE.de(:[0-9]+)?(/.*)?$ [NC]
        RewriteCond %{HTTP_REFERER} !^http://(.*@)?1.2.3.4(:[0-9]+)?(/.*)?$
        RewriteRule .*.(gif|jpeg|jpg)$ - [NC,F,L]
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/EXAMPLE.com-error.log
    CustomLog ${APACHE_LOG_DIR}/EXAMPLE.com-access.log combined
    # sends 404-not-found errors to error page
    ErrorDocument 404 /404-error-page.html

    # makes server side includes work on all html pages
    AddType text/html .shtml .html .htm
    AddHandler server-parsed .shtml .html .htm

    RewriteEngine On

    # If the hostname is NOT www.domain.com
    # RewriteCond %{HTTP_HOST} !^www.EXAMPLE.com$
    # 301 redirect to the same resource on www.EXAMPLE.com
    # RewriteRule (.*) http://www.EXAMPLE.com$1 [L,R=301]

    # sets the web surfer's browser to cache images, style sheets, and JavaScript for a week
    <IfModule mod_headers.c>
    # WEEK
    <FilesMatch ".(jpg|jpeg|png|gif|swf|js|css)$">
        Header set Cache-Control "max-age=604800, public"
    </FilesMatch>
    </IfModule>

</VirtualHost>

Any help anyone can give me will be greatly appreciated! It’s amazing what you all do to help other fledgling programmers like myself. Thank you, thank you, thank you.