How to make mod_ssl pointing to different openssl directory instead of default one in apache?

Question

The question is raised because the original openssl version doesn't pass the Vulnerability Scanning, therefore I need to upgrade the openssl package on apache centos 7 and I am using httpd. I have some observations after spending some time

1. The default openssl version is OpenSSL 1.0.2k-fips, which is the default openssl version for mod_ssl, which in turn is a dependency of httpd 2.4.6-97.el7.centos.5 package, so when I yum install httpd, the mod_ssl is automatically pointing to that openssl version

2. I follow the guideline in https://gist.github.com/fernandoaleman/5459173e24d59b45ae2cfc618e20fe06 , instead of version 1.1.1, I installed version 3.0.0, other commands are the same as the guideline mentioned, I checked the openssl version after the upgrade by running openssl version, the output is correct, but when I send request to the server, the response header still displays the original version: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_wsgi/4.5.18 Python/3.6

3. This is the output when I ldd mod_ssl.so in /usr/lib64/httpd/modules folder:

        linux-vdso.so.1 =>  (0x00007ffd44ddd000)
        libssl.so.10 => /lib64/libssl.so.10 (0x00007fe07f09c000)
        libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007fe07ec39000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe07ea1d000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fe07e819000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fe07e44b000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007fe07e1fe000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fe07df15000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007fe07dd11000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fe07dade000)
        libz.so.1 => /lib64/libz.so.1 (0x00007fe07d8c8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fe07f545000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007fe07d6b8000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007fe07d4b4000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fe07d29a000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fe07d073000)
        libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fe07ce11000)

I know that it's a dependency issue, the mod_ssl is not pointing to the new installed openssl 3.0.0, so is there a way to make mod_ssl point to the new openssl version? or I can install openssl 3.0.0 to the default directory that mod_ssl is pointing to? (I don't want to install httpd manually because I don't want to break the dependencies ) I am a beginner of server stuff, it is appreciated if you guys can give me a hand, thanks

Answer 1

Assuming you already have apache + mod_ssl + openssl installed from the official centos repository, you could do something like this:

For OpenSSL:

cd /opt/
wget https://ftp.openssl.org/source/openssl-1.1.1k.tar.gz
tar zxvf openssl-1.1.1k.tar.gz
cd /opt/openssl-1.1.1k
./config -fpic shared
make -j4
make install

For Apache (you could probably use a newer version, I just took the same version as from the official repository):

cd /opt
wget https://dlcdn.apache.org/httpd/httpd-2.4.53.tar.gz
tar zxvf httpd-2.4.53.tar.gz
cd /opt/httpd-2.4.53
./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/sh--includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --prefix=/etc/httpd --exec-prefix=/usr --bindir=/usr/--sbindir=/usr/sbin --mandir=/usr/share/man --libdir=/usr/lib64 --sysconfdir=/etc/httpd/conf --includedir=/usr/include/httpd --libexecdir=/usr/lib64/httpd/modules --datadir=/usr/share/httpd --enable-layout=Fed--with-installbuilddir=/usr/lib64/httpd/build --enable-mpms-shared=all --with-apr=/usr --with-apr-util=/usr --enable-suexec --with-suexec --enable-suexec-capabilities --with-suexec-caller=apache --with-suexec-oot=/var/--without-suexec-logfile --with-suexec-syslog --with-suexec-bin=/usr/sbin/suexec --with-suexec-uidmin=500 --with-suexec-gidmin=100 --enable-pie --with-pcre --enable-mods-shared=all --enable-ssl --with-ssl --die-distca--enable-proxy --enable-cache --enable-disk-cache --enable-ldap --enable-authnz-ldap --enable-cgid --enable-cgi --enable-authn-anon --enable-authn-alias --disable-imag

make -j4
make install

systemctl restart httpd

Basically it will compile Apache from source, mod_ssl as well and it will replace the default files with the ones you just compiled.

Tested it on my centos7 linux box:

[Tue May 17 12:13:19.777713 2022] [mpm_prefork:notice] [pid 3179] AH00163: Apache/2.4.53 (Unix) OpenSSL/1.1.1k configured -- resuming normal operations

Answer 2

I finally solved this question, it is quite tedious, below is my steps to solve this question:

1. backup all related file, which is very useful in later step, you can first use rpm -ql httpd, rpm -ql mod_ssl, to see all the installed file paths and back up accordingly, of course the main config folder should be considered first (in my case: /etc/httpd), it's better to backup mod_ssl configs and files as well because we need to create ssl.conf by ourselves later and it requires backuped configs and files. Uninstall all httpd related file on the server.
2. Compiling openssl from source, I followed the instructions here: Update Openssl to 3.0 on Centos7, the default installed openssl folder is: /usr/local/ssl
3. Compiling Apache HTTPD from source, you can follow the steps Bogdan Stoica mentioned above (very useful info and I changed the command slightly), use the following command:

./configure 
--build=x86_64-redhat-linux-gnu 
--host=x86_64-redhat-linux-gnu 
--disable-dependency-tracking 
--localstatedir=/var 
--sharedstatedir=/var/lib 
--mandir=/usr/share/man 
--infodir=/usr/share/info 
--prefix=/etc/httpd 
--exec-prefix=/usr 
--bindir=/usr/
--sbindir=/usr/sbin 
--libdir=/usr/lib64 
--sysconfdir=/etc/httpd/conf 
--includedir=/usr/include/httpd
--libexecdir=/usr/lib64/httpd/modules
--datadir=/usr/share/httpd 
--with-installbuilddir=/usr/lib64/httpd/build 
--enable-mpms-shared=all 
--with-apr=/usr 
--with-apr-util=/usr 
--enable-suexec 
--with-suexec 
--enable-suexec-capabilities 
--with-suexec-caller=apache 
--with-suexec-oot=/var/
--without-suexec-logfile 
--with-suexec-syslog 
--with-suexec-bin=/usr/sbin/suexec 
--with-suexec-uidmin=500 
--with-suexec-gidmin=100 
--enable-pie --with-pcre 
--enable-mods-shared=all 
--enable-ssl 
--with-ssl=/usr/local
--enable-proxy 
--enable-cache 
--enable-disk-cache 
--enable-ldap 
--enable-authnz-ldap 
--enable-cgid 
--enable-cgi 
--enable-authn-anon 
--enable-authn-alias 
--disable-imag

Note that it is --with-ssl=/usr/local but not --with-ssl=/usr/local/ssland the paths of installed httpd is not exactly the same as the ones from yum install

4. check if new HTTPD is installed by httpd -v, if not, please stop here.

5. check if mod_ssl.so is pointing to new openssl by using ldd /usr/lib64/httpd/modules/mod_ssl.so, here is the output:

linux-vdso.so.1 =>  (0x00007ffc73775000)
libssl.so.3 => /usr/local/lib64/libssl.so.3 (0x00007f85b9040000)
libcrypto.so.3 => /usr/local/lib64/libcrypto.so.3 (0x00007f85b89d4000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f85b87b8000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f85b85b4000)
libc.so.6 => /lib64/libc.so.6 (0x00007f85b81e6000)
/lib64/ld-linux-x86-64.so.2 (0x00007f85b9536000)

If there is no change compared with the original mod_ssl.so file, please stop here.

6. Edit /etc/httpd/httpd.conf, replaced by the backuped httpd.conf, and see what files are lacking, some cnp actions are needed in this step (in my case, some files related to /conf.d/*.conf)

then it's done, so basically the easy way to make mod_ssl point to different openssl is to compile httpd from source with specified openssl directory, but need to be careful about the backup, if you want a better solution, then I think finding configure which is exactly the same as yum install's one will benefit to the consistency issue.

Answer 3

So, as others have commented (and my feelings align) whenever we have a reasonably amount of processing to do with some JSON thing, it's easier to parse the JSON to objects and work with the objects. I (personally, no affiliation) head over to https://quicktype.io and paste the JSON in there, to generate a set of classes:

// <auto-generated />
//
// To parse this JSON data, add NuGet 'Newtonsoft.Json' then do:
//
//    using HackyTracky;
//
//    var trackyThing = TrackyThing.FromJson(jsonString);

namespace HackyTracky
{
    using System;
    using System.Collections.Generic;

    using System.Globalization;
    using Newtonsoft.Json;
    using Newtonsoft.Json.Converters;

    public partial class TrackyThing
    {
        [JsonProperty("transactionId")]
        public Guid TransactionId { get; set; }

        [JsonProperty("output")]
        public Output Output { get; set; }
    }

    public partial class Output
    {
        [JsonProperty("completeTrackResults")]
        public CompleteTrackResult[] CompleteTrackResults { get; set; }
    }

    public partial class CompleteTrackResult
    {
        [JsonProperty("trackingNumber")]
        public string TrackingNumber { get; set; }

        [JsonProperty("trackResults")]
        public TrackResult[] TrackResults { get; set; }
    }

    public partial class TrackResult
    {
        [JsonProperty("trackingNumberInfo")]
        public TrackingNumberInfo TrackingNumberInfo { get; set; }

        [JsonProperty("additionalTrackingInfo")]
        public AdditionalTrackingInfo AdditionalTrackingInfo { get; set; }

        [JsonProperty("shipperInformation")]
        public Information ShipperInformation { get; set; }

        [JsonProperty("recipientInformation")]
        public Information RecipientInformation { get; set; }

        [JsonProperty("latestStatusDetail")]
        public LatestStatusDetail LatestStatusDetail { get; set; }

        [JsonProperty("dateAndTimes")]
        public DateAndTime[] DateAndTimes { get; set; }

        [JsonProperty("availableImages")]
        public AvailableImage[] AvailableImages { get; set; }

        [JsonProperty("specialHandlings")]
        public SpecialHandling[] SpecialHandlings { get; set; }

        [JsonProperty("packageDetails")]
        public PackageDetails PackageDetails { get; set; }

        [JsonProperty("shipmentDetails")]
        public ShipmentDetails ShipmentDetails { get; set; }

        [JsonProperty("scanEvents")]
        public ScanEvent[] ScanEvents { get; set; }

        [JsonProperty("availableNotifications")]
        public string[] AvailableNotifications { get; set; }

        [JsonProperty("deliveryDetails")]
        public DeliveryDetails DeliveryDetails { get; set; }

        [JsonProperty("originLocation")]
        public OriginLocation OriginLocation { get; set; }

        [JsonProperty("destinationLocation")]
        public DestinationLocation DestinationLocation { get; set; }

        [JsonProperty("lastUpdatedDestinationAddress")]
        public LastUpdatedDestinationAddress LastUpdatedDestinationAddress { get; set; }

        [JsonProperty("serviceDetail")]
        public ServiceDetail ServiceDetail { get; set; }

        [JsonProperty("standardTransitTimeWindow")]
        public StandardTransitTimeWindow StandardTransitTimeWindow { get; set; }

        [JsonProperty("estimatedDeliveryTimeWindow")]
        public EstimatedDeliveryTimeWindow EstimatedDeliveryTimeWindow { get; set; }

        [JsonProperty("goodsClassificationCode")]
        public string GoodsClassificationCode { get; set; }

        [JsonProperty("returnDetail")]
        public ReturnDetail ReturnDetail { get; set; }
    }

    public partial class AdditionalTrackingInfo
    {
        [JsonProperty("nickname")]
        public string Nickname { get; set; }

        [JsonProperty("packageIdentifiers")]
        public PackageIdentifier[] PackageIdentifiers { get; set; }

        [JsonProperty("hasAssociatedShipments")]
        public bool HasAssociatedShipments { get; set; }
    }

    public partial class PackageIdentifier
    {
        [JsonProperty("type")]
        public string Type { get; set; }

        [JsonProperty("values")]
        public string[] Values { get; set; }

        [JsonProperty("trackingNumberUniqueId")]
        public string TrackingNumberUniqueId { get; set; }

        [JsonProperty("carrierCode")]
        public string CarrierCode { get; set; }
    }

    public partial class AvailableImage
    {
        [JsonProperty("type")]
        public string Type { get; set; }
    }

    public partial class DateAndTime
    {
        [JsonProperty("type")]
        public string Type { get; set; }

        [JsonProperty("dateTime")]
        public DateTimeOffset DateTime { get; set; }
    }

    public partial class DeliveryDetails
    {
        [JsonProperty("actualDeliveryAddress")]
        public LastUpdatedDestinationAddress ActualDeliveryAddress { get; set; }

        [JsonProperty("locationType")]
        public string LocationType { get; set; }

        [JsonProperty("locationDescription")]
        public string LocationDescription { get; set; }

        [JsonProperty("deliveryAttempts")]
        [JsonConverter(typeof(ParseStringConverter))]
        public long DeliveryAttempts { get; set; }

        [JsonProperty("receivedByName")]
        public string ReceivedByName { get; set; }

        [JsonProperty("deliveryOptionEligibilityDetails")]
        public DeliveryOptionEligibilityDetail[] DeliveryOptionEligibilityDetails { get; set; }
    }

    public partial class LastUpdatedDestinationAddress
    {
        [JsonProperty("city")]
        public string City { get; set; }

        [JsonProperty("stateOrProvinceCode")]
        public string StateOrProvinceCode { get; set; }

        [JsonProperty("countryCode")]
        public string CountryCode { get; set; }

        [JsonProperty("residential")]
        public bool Residential { get; set; }

        [JsonProperty("countryName")]
        public string CountryName { get; set; }
    }

    public partial class DeliveryOptionEligibilityDetail
    {
        [JsonProperty("option")]
        public string Option { get; set; }

        [JsonProperty("eligibility")]
        public string Eligibility { get; set; }
    }

    public partial class DestinationLocation
    {
        [JsonProperty("locationContactAndAddress")]
        public LocationContactAndAddress LocationContactAndAddress { get; set; }

        [JsonProperty("locationType")]
        public string LocationType { get; set; }
    }

    public partial class LocationContactAndAddress
    {
        [JsonProperty("address")]
        public LastUpdatedDestinationAddress Address { get; set; }
    }

    public partial class EstimatedDeliveryTimeWindow
    {
        [JsonProperty("window")]
        public ReturnDetail Window { get; set; }
    }

    public partial class ReturnDetail
    {
    }

    public partial class LatestStatusDetail
    {
        [JsonProperty("code")]
        public string Code { get; set; }

        [JsonProperty("derivedCode")]
        public string DerivedCode { get; set; }

        [JsonProperty("statusByLocale")]
        public string StatusByLocale { get; set; }

        [JsonProperty("description")]
        public string Description { get; set; }

        [JsonProperty("scanLocation")]
        public LastUpdatedDestinationAddress ScanLocation { get; set; }
    }

    public partial class OriginLocation
    {
        [JsonProperty("locationContactAndAddress")]
        public LocationContactAndAddress LocationContactAndAddress { get; set; }

        [JsonProperty("locationId")]
        public string LocationId { get; set; }
    }

    public partial class PackageDetails
    {
        [JsonProperty("packagingDescription")]
        public PackagingDescription PackagingDescription { get; set; }

        [JsonProperty("sequenceNumber")]
        [JsonConverter(typeof(ParseStringConverter))]
        public long SequenceNumber { get; set; }

        [JsonProperty("count")]
        [JsonConverter(typeof(ParseStringConverter))]
        public long Count { get; set; }

        [JsonProperty("weightAndDimensions")]
        public WeightAndDimensions WeightAndDimensions { get; set; }

        [JsonProperty("packageContent")]
        public object[] PackageContent { get; set; }
    }

    public partial class PackagingDescription
    {
        [JsonProperty("type")]
        public string Type { get; set; }

        [JsonProperty("description")]
        public string Description { get; set; }
    }

    public partial class WeightAndDimensions
    {
        [JsonProperty("weight")]
        public Weight[] Weight { get; set; }

        [JsonProperty("dimensions")]
        public Dimension[] Dimensions { get; set; }
    }

    public partial class Dimension
    {
        [JsonProperty("length")]
        public long Length { get; set; }

        [JsonProperty("width")]
        public long Width { get; set; }

        [JsonProperty("height")]
        public long Height { get; set; }

        [JsonProperty("units")]
        public string Units { get; set; }
    }

    public partial class Weight
    {
        [JsonProperty("value")]
        public string Value { get; set; }

        [JsonProperty("unit")]
        public string Unit { get; set; }
    }

    public partial class Information
    {
        [JsonProperty("contact")]
        public ReturnDetail Contact { get; set; }

        [JsonProperty("address")]
        public LastUpdatedDestinationAddress Address { get; set; }
    }

    public partial class ScanEvent
    {
        [JsonProperty("date")]
        public DateTimeOffset Date { get; set; }

        [JsonProperty("eventType")]
        public string EventType { get; set; }

        [JsonProperty("eventDescription")]
        public string EventDescription { get; set; }

        [JsonProperty("exceptionCode")]
        public string ExceptionCode { get; set; }

        [JsonProperty("exceptionDescription")]
        public string ExceptionDescription { get; set; }

        [JsonProperty("scanLocation")]
        public ScanLocation ScanLocation { get; set; }

        [JsonProperty("locationId", NullValueHandling = NullValueHandling.Ignore)]
        public string LocationId { get; set; }

        [JsonProperty("locationType")]
        public string LocationType { get; set; }

        [JsonProperty("derivedStatusCode")]
        public string DerivedStatusCode { get; set; }

        [JsonProperty("derivedStatus")]
        public string DerivedStatus { get; set; }
    }

    public partial class ScanLocation
    {
        [JsonProperty("streetLines")]
        public string[] StreetLines { get; set; }

        [JsonProperty("city", NullValueHandling = NullValueHandling.Ignore)]
        public string City { get; set; }

        [JsonProperty("stateOrProvinceCode", NullValueHandling = NullValueHandling.Ignore)]
        public string StateOrProvinceCode { get; set; }

        [JsonProperty("postalCode", NullValueHandling = NullValueHandling.Ignore)]
        [JsonConverter(typeof(ParseStringConverter))]
        public long? PostalCode { get; set; }

        [JsonProperty("countryCode", NullValueHandling = NullValueHandling.Ignore)]
        public string CountryCode { get; set; }

        [JsonProperty("residential")]
        public bool Residential { get; set; }

        [JsonProperty("countryName", NullValueHandling = NullValueHandling.Ignore)]
        public string CountryName { get; set; }
    }

    public partial class ServiceDetail
    {
        [JsonProperty("type")]
        public string Type { get; set; }

        [JsonProperty("description")]
        public string Description { get; set; }

        [JsonProperty("shortDescription")]
        public string ShortDescription { get; set; }
    }

    public partial class ShipmentDetails
    {
        [JsonProperty("possessionStatus")]
        public bool PossessionStatus { get; set; }

        [JsonProperty("weight")]
        public Weight[] Weight { get; set; }
    }

    public partial class SpecialHandling
    {
        [JsonProperty("type")]
        public string Type { get; set; }

        [JsonProperty("description")]
        public string Description { get; set; }

        [JsonProperty("paymentType")]
        public string PaymentType { get; set; }
    }

    public partial class StandardTransitTimeWindow
    {
        [JsonProperty("window")]
        public Window Window { get; set; }
    }

    public partial class Window
    {
        [JsonProperty("ends")]
        public DateTimeOffset Ends { get; set; }
    }

    public partial class TrackingNumberInfo
    {
        [JsonProperty("trackingNumber")]
        public string TrackingNumber { get; set; }

        [JsonProperty("trackingNumberUniqueId")]
        public string TrackingNumberUniqueId { get; set; }

        [JsonProperty("carrierCode")]
        public string CarrierCode { get; set; }
    }

    public partial class TrackyThing
    {
        public static TrackyThing FromJson(string json) => JsonConvert.DeserializeObject<TrackyThing>(json, HackyTracky.Converter.Settings);
    }

    public static class Serialize
    {
        public static string ToJson(this TrackyThing self) => JsonConvert.SerializeObject(self, HackyTracky.Converter.Settings);
    }

    internal static class Converter
    {
        public static readonly JsonSerializerSettings Settings = new JsonSerializerSettings
        {
            MetadataPropertyHandling = MetadataPropertyHandling.Ignore,
            DateParseHandling = DateParseHandling.None,
            Converters =
            {
                new IsoDateTimeConverter { DateTimeStyles = DateTimeStyles.AssumeUniversal }
            },
        };
    }

    internal class ParseStringConverter : JsonConverter
    {
        public override bool CanConvert(Type t) => t == typeof(long) || t == typeof(long?);

        public override object ReadJson(JsonReader reader, Type t, object existingValue, JsonSerializer serializer)
        {
            if (reader.TokenType == JsonToken.Null) return null;
            var value = serializer.Deserialize<string>(reader);
            long l;
            if (Int64.TryParse(value, out l))
            {
                return l;
            }
            throw new Exception("Cannot unmarshal type long");
        }

        public override void WriteJson(JsonWriter writer, object untypedValue, JsonSerializer serializer)
        {
            if (untypedValue == null)
            {
                serializer.Serialize(writer, null);
                return;
            }
            var value = (long)untypedValue;
            serializer.Serialize(writer, value.ToString());
            return;
        }

        public static readonly ParseStringConverter Singleton = new ParseStringConverter();
    }
}

There are other services that do the same thing - json2csharp is popular too

After you've done this, you can just convert your objects like it says in the comment at the top of the classes:

var trackyThing = TrackyThing.FromJson(jsonString);

And access things via LINQ:

var allScanEvents = trackyThing.Output.CompleteTrackResults
  .SelectMany(ctr => ctr.TrackResults)
  .SelectMany(tr => tr.ScanEvents);