'Parameter estimation in logistic model by negative log-likelihood minimization - R

I am currently attempting to estimate the parameters of a logistic regression model "by hand" on the iris dataset via minimisation of cross-entropy. Please note, when I say iris dataset, it has been changed such that there are only two classes - Setosa and Other. It was also normalised via the scale function:

library(dplyr)
library(optimx)
iriso <- iris %>%
  mutate(Species = ifelse(Species == "setosa", "setosa", "other")) %>%
  mutate(Species_n = ifelse(Species == "setosa", 1, 0)) %>%
  as.data.frame()

iriso[,1:4] <- scale(iriso[,1:4])

Based on my understanding: should everything be correct, when the optimisation is completed we should obtain the same set of parameters each time - no matter the starting point for the optimisation algorithm. However, the parameter estimates following optimisation are bouncing around:

Some functions being defined:

X <- model.matrix(~.,data=iriso[,1:4])
Y <- model.matrix(~0+Species_n,data=iriso)

e <- exp(1)
sigmoid <- function(y){
  1/(1 + e^-y)
}

#w is an array of weights. x is matrix of observations
logistique <- function(w, x){
  sigmoid(
    y = w[1]*x[,1] + w[2]*x[,2] + w[3]*x[,3] + w[4]*x[,4] + w[5]*x[,5]
    )
}

#y is obsrved values
entropie <- function(w, y, x){
  prob_pred <- logistique(w = w, x = x)

  -sum(
    y*log(prob_pred) + (1-y)*log(1-prob_pred)
  ) 
}

Optimisation step:

for(i in 1:5){
  w0 <- rnorm(n = 5) #set of initial parameters
  optimx(par = w0, fn = entropie,  
         method = "Nelder-Mead",
         y = iriso$Species_n, x = X) %>%
    print()
}

I can not seem to understand why I am not obtaining consistent answers. Is there something wrong with the code above? Is there a concept I am not aware of? Something I missed?

Thanks.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source