Is it possible to avoid the build dir being added as library/framework search dir in Xcode?

Question

I have a rare issue with the default behaviour of Xcode (which I guess only causes issues in super specific cases like mine) - I have an umbrella framework which links to a framework with the same name thus after the first build ld always issues the is linking with itself warning and messes up the actual link dependencies. This happens since Xcode seems to add the build dir as a library/framework search path with -L and -F flags (see sample image below). This is not due to the "Framework Search Paths" / "Library Search Paths" options, since they are empty in Xcode settings. Does anyone know if there is something else - some other option/flag in the Xcode settings which triggers this in an inexplicit way that I have overseen? As an alternative is there a way to negate/ignore the first search paths with later arguments that I could add myself?

For example making an empty Swift project produces the same behaviour - the build dir is added as a library/framework search path before all other arguments.