Adding a JPanel to a JScrollPane doesn't work properly, where is the mistake?

Question

I have the following problem:

I want to add a JScrollPane to a JPanel, nothing special actually. When I start the GUI without the scroll pane, everything works fine, the GUI shows every component that I have implemented. I have a JFrame, inside it (BorderLayout.CENTER) lies a panel framePanel and inside that panel there are 3 panels, that are located at BorderLayout.NORTH, CENTER and SOUTH.

As soon as I add the framePanel to a scroll pane and the scroll pane to the frame, instead of the framePanel, the GUI doesn't show the components anymore (only sometimes, 1 out of 7). I will show you the relevant code where I didn't add scroll pane, and where I did add scroll pane.

/**
 * In this method, the standard settings for the frame will be set and
 * it will add the framePanel, which contains the JScrollPane
 */
public void setFrameSettings() {
    frame.setTitle("Spielekoffer");
    frame.setSize(new Dimension(1035, 750));
    frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
    frame.setVisible(true);
    frame.setResizable(false);
    
    // BorderLayout on the top-level-container
    frame.setLayout(new BorderLayout());
    
    // BorderLayout from the panel inside the frame
    framePanel.setLayout(new BorderLayout());
    
    // Adding the JScrollPane
    JScrollPane scroller = new JScrollPane(framePanel);
    frame.add(scroller, BorderLayout.CENTER);
}

Now the scroll pane is added, and then it doesn't work properly, but if I make it like this it works:

    // NOT adding the JScrollPane
    //JScrollPane scroller = new JScrollPane(framePanel);
    frame.add(framePanel, BorderLayout.CENTER);