how to make my pickup script not make gameobjects go through other objects?

Question

me and my friend are making a game and we need a script to pickup rigidbodys in the scene

and we found one but it doesnt work so we made one ourselfs but we are having some problems

so we are using rb.constraints = RigidbodyConstraints.FreezeAll; and setting the objects transform to a HeldParent thats in front of the player but the held object goes through the walls and floor and interacts poorly with other objects in the scene

is there a way so it doesnt do this?

heres our current code:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Grab : MonoBehaviour
{
    public Transform HeldParent;
    private Rigidbody rb;
    bool Grabbing;

    void OnMouseDown()
    {

        Grabbing = true;
        rb = GetComponent<Rigidbody>();
        GetComponent<Rigidbody>().useGravity = false;
        this.transform.position = HeldParent.position;
        this.transform.parent = GameObject.Find("HeldParent").transform;
        rb.constraints = RigidbodyConstraints.FreezeAll;
    }
    void OnMouseUp()
    {
        Grabbing = false;
        this.transform.parent = null;
        GetComponent<Rigidbody>().useGravity = true;
        rb.constraints = RigidbodyConstraints.None;
    }
}