'.htaccess Rule blocks Bots from crawling?

I am currently working on a project and I can't get the sitemap uploaded to GSC because its not finding the Robots.txt file and gives me the error code: (503 Service Unavailable)

I havn't been to deep into .htaccess. Might someone also explain me the usage of it? And now to the question. When I look into it the first lines are:

    #<mittwald 503 for bots>
###########503-for-Bots##########################
#################################################

SetEnvIfNoCase User-Agent "Googlebot" bots
#SetEnvIfNoCase User-Agent "Googlebot-Image" bots
#SetEnvIfNoCase User-Agent "Googlebot-Mobile" bots
#SetEnvIfNoCase User-Agent "Java" bots
#SetEnvIfNoCase User-Agent "UptimeRobot" bots
SetEnvIfNoCase User-Agent "bingbot" bots
#SetEnvIfNoCase User-Agent "msnbot" bots
#SetEnvIfNoCase User-Agent "msnbot-media" bots
SetEnvIfNoCase User-Agent "AACrawler" bots
SetEnvIfNoCase User-Agent "AcoonBot" bots
SetEnvIfNoCase User-Agent "AhrefsBot" bots
SetEnvIfNoCase User-Agent "BLEXBot" bots
SetEnvIfNoCase User-Agent "BUbiNG" bots
SetEnvIfNoCase User-Agent "Baiduspider" bots
SetEnvIfNoCase User-Agent "Baiduspider-image" bots
SetEnvIfNoCase User-Agent "CCBot" bots
SetEnvIfNoCase User-Agent "CaBot" bots
SetEnvIfNoCase User-Agent "CompSpyBot" bots
SetEnvIfNoCase User-Agent "Daum" bots
SetEnvIfNoCase User-Agent "DoCoMo" bots
SetEnvIfNoCase User-Agent "DotBot" bots
SetEnvIfNoCase User-Agent "Exabot" bots
SetEnvIfNoCase User-Agent "Ezooms" bots
SetEnvIfNoCase User-Agent "Faraday" bots
SetEnvIfNoCase User-Agent "FeedBurner" bots
SetEnvIfNoCase User-Agent "HuaweiSymantecSpider" bots
SetEnvIfNoCase User-Agent "Infohelfer" bots
SetEnvIfNoCase User-Agent "JobboerseBot" bots
SetEnvIfNoCase User-Agent "Jooblebot" bots
SetEnvIfNoCase User-Agent "Landau-Media-Spider" bots
SetEnvIfNoCase User-Agent "MJ12bot" bots
SetEnvIfNoCase User-Agent "Mail.RU_Bot" bots
SetEnvIfNoCase User-Agent "MegaIndex" bots
SetEnvIfNoCase User-Agent "MegaIndex.ru" bots
SetEnvIfNoCase User-Agent "MetaJobBot" bots
SetEnvIfNoCase User-Agent "MetricsBot" bots
SetEnvIfNoCase User-Agent "MojeekBot" bots
SetEnvIfNoCase User-Agent "Nutch-1.10" bots
SetEnvIfNoCase User-Agent "OnPageBot" bots
SetEnvIfNoCase User-Agent "OpenWebSpider" bots
SetEnvIfNoCase User-Agent "PRTGCloudBot" bots
SetEnvIfNoCase User-Agent "PiplBot" bots
SetEnvIfNoCase User-Agent "Pixray-Seeker" bots
SetEnvIfNoCase User-Agent "Qwantify" bots
SetEnvIfNoCase User-Agent "Ralocobot" bots
SetEnvIfNoCase User-Agent "SEOkicks" bots
SetEnvIfNoCase User-Agent "SEOkicks-Robot" bots
SetEnvIfNoCase User-Agent "Screaming" bots
SetEnvIfNoCase User-Agent "Seekport" bots
SetEnvIfNoCase User-Agent "SemrushBot" bots
SetEnvIfNoCase User-Agent "SeznamBot" bots
SetEnvIfNoCase User-Agent "Site24x7" bots
SetEnvIfNoCase User-Agent "Slurp" bots
SetEnvIfNoCase User-Agent "Sosospider" bots
SetEnvIfNoCase User-Agent "Speedy" bots
SetEnvIfNoCase User-Agent "Spider" bots
SetEnvIfNoCase User-Agent "Spiderlytics" bots
SetEnvIfNoCase User-Agent "Superfeedr" bots
SetEnvIfNoCase User-Agent "Toweya.com" bots
SetEnvIfNoCase User-Agent "TurnitinBot" bots
SetEnvIfNoCase User-Agent "TwengaBot" bots
SetEnvIfNoCase User-Agent "Twiceler" bots
SetEnvIfNoCase User-Agent "Twitterbot" bots
SetEnvIfNoCase User-Agent "UnisterBot" bots
SetEnvIfNoCase User-Agent "WebCapture" bots
SetEnvIfNoCase User-Agent "Yandex" bots
SetEnvIfNoCase User-Agent "ZumBot" bots
SetEnvIfNoCase User-Agent "adscanner" bots
SetEnvIfNoCase User-Agent "aggregator:Spinn3r" bots
SetEnvIfNoCase User-Agent "crawler" bots
SetEnvIfNoCase User-Agent "[email protected]" bots
SetEnvIfNoCase User-Agent "domaincrawler.com" bots
SetEnvIfNoCase User-Agent "echobot" bots
SetEnvIfNoCase User-Agent "heritrix" bots
SetEnvIfNoCase User-Agent "htdig" bots
SetEnvIfNoCase User-Agent "ia_archiver" bots
SetEnvIfNoCase User-Agent "imaBot" bots
SetEnvIfNoCase User-Agent "infometrics-bot" bots
SetEnvIfNoCase User-Agent "jobs.de-Robot" bots
SetEnvIfNoCase User-Agent "linkdex.com" bots
SetEnvIfNoCase User-Agent "magpie-crawler" bots
SetEnvIfNoCase User-Agent "meanpathbot" bots
SetEnvIfNoCase User-Agent "pricebot" bots
SetEnvIfNoCase User-Agent "psbot" bots
SetEnvIfNoCase User-Agent "rogerbot" bots
SetEnvIfNoCase User-Agent "seoscanners" bots
SetEnvIfNoCase User-Agent "smtbot" bots
SetEnvIfNoCase User-Agent "spbot" bots
SetEnvIfNoCase User-Agent "spider" bots
SetEnvIfNoCase User-Agent "ssearch_bot" bots
SetEnvIfNoCase User-Agent "tb-webde" bots
SetEnvIfNoCase User-Agent "textlink_status_crawler" bots
SetEnvIfNoCase User-Agent "tracemyfile" bots
SetEnvIfNoCase User-Agent "trendictionbot" bots
SetEnvIfNoCase User-Agent "vebidoobot-Robot" bots
#### Retry 12h
Header always set Retry-After "43200" env=bots
RewriteEngine on
RewriteCond %{ENV:bots} 1
RewriteRule .* - [R=503,L]
################################################
################################################
#</mittwald 503 for bots>

# Disallow access to important files
<FilesMatch "(^\.|wp-config\.php|xmlrpc\.php|(?<!robots)\.txt|(liesmich|readme)\.*)">
Require all denied
</FilesMatch>

# ----------------------------------------------------------------------
# | Compression                                                        |
# ----------------------------------------------------------------------

<IfModule mod_deflate.c>

    # Force compression for mangled `Accept-Encoding` request headers
    # https://developer.yahoo.com/blogs/ydn/pushing-beyond-gzipping-25601.html

    <IfModule mod_setenvif.c>
        <IfModule mod_headers.c>
            SetEnvIfNoCase ^(Accept-EncodXng|X-cept-Encoding|X{15}|~{15}|-{15})$ ^((gzip|deflate)\s*,?\s*)+|[X~-]{4,13}$ HAVE_Accept-Encoding
            RequestHeader append Accept-Encoding "gzip,deflate" env=HAVE_Accept-Encoding
        </IfModule>
    </IfModule>

    # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

    # Compress all output labeled with one of the following media types.
    #
    # (!) For Apache versions below version 2.3.7 you don't need to
    # enable `mod_filter` and can remove the `<IfModule mod_filter.c>`
    # and `</IfModule>` lines as `AddOutputFilterByType` is still in
    # the core directives.
    #
    # https://httpd.apache.org/docs/current/mod/mod_filter.html#addoutputfilterbytype

    <IfModule mod_filter.c>
        AddOutputFilterByType DEFLATE "application/atom+xml" \
                                      "application/javascript" \
                                      "application/json" \
                                      "application/ld+json" \
                                      "application/manifest+json" \
                                      "application/rdf+xml" \
                                      "application/rss+xml" \
                                      "application/schema+json" \
                                      "application/vnd.geo+json" \
                                      "application/vnd.ms-fontobject" \
                                      "application/x-font-ttf" \
                                      "application/x-javascript" \
                                      "application/x-web-app-manifest+json" \
                                      "application/xhtml+xml" \
                                      "application/xml" \
                                      "font/collection" \
                                      "font/eot" \
                                      "font/opentype" \
                                      "font/otf" \
                                      "font/ttf" \
                                      "image/bmp" \
                                      "image/svg+xml" \
                                      "image/vnd.microsoft.icon" \
                                      "image/x-icon" \
                                      "text/cache-manifest" \
                                      "text/calendar" \
                                      "text/css" \
                                      "text/html" \
                                      "text/javascript" \
                                      "text/plain" \
                                      "text/markdown" \
                                      "text/vcard" \
                                      "text/vnd.rim.location.xloc" \
                                      "text/vtt" \
                                      "text/x-component" \
                                      "text/x-cross-domain-policy" \
                                      "text/xml"

    </IfModule>

    # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

    # Map the following filename extensions to the specified
    # encoding type in order to make Apache serve the file types
    # with the appropriate `Content-Encoding` response header
    # (do note that this will NOT make Apache compress them!).
    #
    # If these files types would be served without an appropriate
    # `Content-Enable` response header, client applications (e.g.:
    # browsers) wouldn't know that they first need to uncompress
    # the response, and thus, wouldn't be able to understand the
    # content.
    #
    # https://httpd.apache.org/docs/current/mod/mod_mime.html#addencoding

    <IfModule mod_mime.c>
        AddEncoding gzip              svgz
    </IfModule>

</IfModule>

# ----------------------------------------------------------------------
# | Expires headers                                                    |
# ----------------------------------------------------------------------

# Serve resources with far-future expires headers.
#
# (!) If you don't control versioning with filename-based
# cache busting, you should consider lowering the cache times
# to something like one week.
#
# https://httpd.apache.org/docs/current/mod/mod_expires.html

<IfModule mod_expires.c>

    ExpiresActive on
    ExpiresDefault                                      "access plus 1 month"

  # CSS

    ExpiresByType text/css                              "access plus 1 year"


  # Data interchange

    ExpiresByType application/atom+xml                  "access plus 1 hour"
    ExpiresByType application/rdf+xml                   "access plus 1 hour"
    ExpiresByType application/rss+xml                   "access plus 1 hour"

    ExpiresByType application/json                      "access plus 0 seconds"
    ExpiresByType application/ld+json                   "access plus 0 seconds"
    ExpiresByType application/schema+json               "access plus 0 seconds"
    ExpiresByType application/vnd.geo+json              "access plus 0 seconds"
    ExpiresByType application/xml                       "access plus 0 seconds"
    ExpiresByType text/calendar                         "access plus 0 seconds"
    ExpiresByType text/xml                              "access plus 0 seconds"


  # Favicon (cannot be renamed!) and cursor images

    ExpiresByType image/vnd.microsoft.icon              "access plus 1 week"
    ExpiresByType image/x-icon                          "access plus 1 week"

  # HTML

    ExpiresByType text/html                             "access plus 0 seconds"


  # JavaScript

    ExpiresByType application/javascript                "access plus 1 year"
    ExpiresByType application/x-javascript              "access plus 1 year"
    ExpiresByType text/javascript                       "access plus 1 year"


  # Manifest files

    ExpiresByType application/manifest+json             "access plus 1 week"
    ExpiresByType application/x-web-app-manifest+json   "access plus 0 seconds"
    ExpiresByType text/cache-manifest                   "access plus 0 seconds"


  # Markdown

    ExpiresByType text/markdown                         "access plus 0 seconds"


  # Media files

    ExpiresByType audio/ogg                             "access plus 1 month"
    ExpiresByType image/bmp                             "access plus 1 month"
    ExpiresByType image/gif                             "access plus 1 month"
    ExpiresByType image/jpeg                            "access plus 1 month"
    ExpiresByType image/png                             "access plus 1 month"
    ExpiresByType image/svg+xml                         "access plus 1 month"
    ExpiresByType image/webp                            "access plus 1 month"
    ExpiresByType video/mp4                             "access plus 1 month"
    ExpiresByType video/ogg                             "access plus 1 month"
    ExpiresByType video/webm                            "access plus 1 month"


  # Web fonts

    # Collection
    ExpiresByType font/collection                       "access plus 1 month"

    # Embedded OpenType (EOT)
    ExpiresByType application/vnd.ms-fontobject         "access plus 1 month"
    ExpiresByType font/eot                              "access plus 1 month"

    # OpenType
    ExpiresByType font/opentype                         "access plus 1 month"
    ExpiresByType font/otf                              "access plus 1 month"

    # TrueType
    ExpiresByType application/x-font-ttf                "access plus 1 month"
    ExpiresByType font/ttf                              "access plus 1 month"

    # Web Open Font Format (WOFF) 1.0
    ExpiresByType application/font-woff                 "access plus 1 month"
    ExpiresByType application/x-font-woff               "access plus 1 month"
    ExpiresByType font/woff                             "access plus 1 month"

    # Web Open Font Format (WOFF) 2.0
    ExpiresByType application/font-woff2                "access plus 1 month"
    ExpiresByType font/woff2                            "access plus 1 month"


  # Other

    ExpiresByType text/x-cross-domain-policy            "access plus 1 week"

</IfModule>

# BEGIN W3TC Browser Cache
<IfModule mod_mime.c>
    AddType text/css .css
    AddType text/x-component .htc
    AddType application/x-javascript .js
    AddType application/javascript .js2
    AddType text/javascript .js3
    AddType text/x-js .js4
    AddType text/html .html .htm
    AddType text/richtext .rtf .rtx
    AddType image/svg+xml .svg
    AddType text/plain .txt
    AddType text/xsd .xsd
    AddType text/xsl .xsl
    AddType text/xml .xml
    AddType video/asf .asf .asx .wax .wmv .wmx
    AddType video/avi .avi
    AddType image/bmp .bmp
    AddType application/java .class
    AddType video/divx .divx
    AddType application/msword .doc .docx
    AddType application/vnd.ms-fontobject .eot
    AddType application/x-msdownload .exe
    AddType image/gif .gif
    AddType application/x-gzip .gz .gzip
    AddType image/x-icon .ico
    AddType image/jpeg .jpg .jpeg .jpe
    AddType image/webp .webp
    AddType application/json .json
    AddType application/vnd.ms-access .mdb
    AddType audio/midi .mid .midi
    AddType video/quicktime .mov .qt
    AddType audio/mpeg .mp3 .m4a
    AddType video/mp4 .mp4 .m4v
    AddType video/mpeg .mpeg .mpg .mpe
    AddType application/vnd.ms-project .mpp
    AddType application/x-font-otf .otf
    AddType application/vnd.ms-opentype ._otf
    AddType application/vnd.oasis.opendocument.database .odb
    AddType application/vnd.oasis.opendocument.chart .odc
    AddType application/vnd.oasis.opendocument.formula .odf
    AddType application/vnd.oasis.opendocument.graphics .odg
    AddType application/vnd.oasis.opendocument.presentation .odp
    AddType application/vnd.oasis.opendocument.spreadsheet .ods
    AddType application/vnd.oasis.opendocument.text .odt
    AddType audio/ogg .ogg
    AddType application/pdf .pdf
    AddType image/png .png
    AddType application/vnd.ms-powerpoint .pot .pps .ppt .pptx
    AddType audio/x-realaudio .ra .ram
    AddType image/svg+xml .svg .svgz
    AddType application/x-shockwave-flash .swf
    AddType application/x-tar .tar
    AddType image/tiff .tif .tiff
    AddType application/x-font-ttf .ttf .ttc
    AddType application/vnd.ms-opentype ._ttf
    AddType audio/wav .wav
    AddType audio/wma .wma
    AddType application/vnd.ms-write .wri
    AddType application/font-woff .woff
    AddType application/font-woff2 .woff2
    AddType application/vnd.ms-excel .xla .xls .xlsx .xlt .xlw
    AddType application/zip .zip
</IfModule>
<IfModule mod_expires.c>
    ExpiresActive On
    ExpiresByType text/css A31536000
    ExpiresByType text/x-component A31536000
    ExpiresByType application/x-javascript A31536000
    ExpiresByType application/javascript A31536000
    ExpiresByType text/javascript A31536000
    ExpiresByType text/x-js A31536000
    ExpiresByType text/html A3600
    ExpiresByType text/richtext A3600
    ExpiresByType image/svg+xml A3600
    ExpiresByType text/plain A3600
    ExpiresByType text/xsd A3600
    ExpiresByType text/xsl A3600
    ExpiresByType text/xml A3600
    ExpiresByType video/asf A31536000
    ExpiresByType video/avi A31536000
    ExpiresByType image/bmp A31536000
    ExpiresByType application/java A31536000
    ExpiresByType video/divx A31536000
    ExpiresByType application/msword A31536000
    ExpiresByType application/vnd.ms-fontobject A31536000
    ExpiresByType application/x-msdownload A31536000
    ExpiresByType image/gif A31536000
    ExpiresByType application/x-gzip A31536000
    ExpiresByType image/x-icon A31536000
    ExpiresByType image/jpeg A31536000
    ExpiresByType image/webp A31536000
    ExpiresByType application/json A31536000
    ExpiresByType application/vnd.ms-access A31536000
    ExpiresByType audio/midi A31536000
    ExpiresByType video/quicktime A31536000
    ExpiresByType audio/mpeg A31536000
    ExpiresByType video/mp4 A31536000
    ExpiresByType video/mpeg A31536000
    ExpiresByType application/vnd.ms-project A31536000
    ExpiresByType application/x-font-otf A31536000
    ExpiresByType application/vnd.ms-opentype A31536000
    ExpiresByType application/vnd.oasis.opendocument.database A31536000
    ExpiresByType application/vnd.oasis.opendocument.chart A31536000
    ExpiresByType application/vnd.oasis.opendocument.formula A31536000
    ExpiresByType application/vnd.oasis.opendocument.graphics A31536000
    ExpiresByType application/vnd.oasis.opendocument.presentation A31536000
    ExpiresByType application/vnd.oasis.opendocument.spreadsheet A31536000
    ExpiresByType application/vnd.oasis.opendocument.text A31536000
    ExpiresByType audio/ogg A31536000
    ExpiresByType application/pdf A31536000
    ExpiresByType image/png A31536000
    ExpiresByType application/vnd.ms-powerpoint A31536000
    ExpiresByType audio/x-realaudio A31536000
    ExpiresByType image/svg+xml A31536000
    ExpiresByType application/x-shockwave-flash A31536000
    ExpiresByType application/x-tar A31536000
    ExpiresByType image/tiff A31536000
    ExpiresByType application/x-font-ttf A31536000
    ExpiresByType application/vnd.ms-opentype A31536000
    ExpiresByType audio/wav A31536000
    ExpiresByType audio/wma A31536000
    ExpiresByType application/vnd.ms-write A31536000
    ExpiresByType application/font-woff A31536000
    ExpiresByType application/font-woff2 A31536000
    ExpiresByType application/vnd.ms-excel A31536000
    ExpiresByType application/zip A31536000
</IfModule>
<IfModule mod_deflate.c>
        AddOutputFilterByType DEFLATE text/css text/x-component application/x-javascript application/javascript text/javascript text/x-js text/html text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/bmp application/java application/msword application/vnd.ms-fontobject application/x-msdownload image/x-icon image/webp application/json application/vnd.ms-access application/vnd.ms-project application/x-font-otf application/vnd.ms-opentype application/vnd.oasis.opendocument.database application/vnd.oasis.opendocument.chart application/vnd.oasis.opendocument.formula application/vnd.oasis.opendocument.graphics application/vnd.oasis.opendocument.presentation application/vnd.oasis.opendocument.spreadsheet application/vnd.oasis.opendocument.text audio/ogg application/pdf application/vnd.ms-powerpoint image/svg+xml application/x-shockwave-flash image/tiff application/x-font-ttf application/vnd.ms-opentype audio/wav application/vnd.ms-write application/font-woff application/font-woff2 application/vnd.ms-excel
    <IfModule mod_mime.c>
        # DEFLATE by extension
        AddOutputFilter DEFLATE js css htm html xml
    </IfModule>
</IfModule>
<FilesMatch "\.(css|htc|less|js|js2|js3|js4|CSS|HTC|LESS|JS|JS2|JS3|JS4)$">
    FileETag MTime Size
    <IfModule mod_headers.c>
        Header set Pragma "public"
        Header append Cache-Control "public"
         Header unset Set-Cookie
         Header set X-Powered-By "W3 Total Cache/0.9.7"
    </IfModule>
</FilesMatch>
<FilesMatch "\.(html|htm|rtf|rtx|svg|txt|xsd|xsl|xml|HTML|HTM|RTF|RTX|SVG|TXT|XSD|XSL|XML)$">
    FileETag MTime Size
    <IfModule mod_headers.c>
        Header append Vary User-Agent env=!dont-vary
        Header set Pragma "public"
        Header append Cache-Control "public"
         Header set X-Powered-By "W3 Total Cache/0.9.7"
    </IfModule>
</FilesMatch>
<FilesMatch "\.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|eot|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|webp|json|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|otf|_otf|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|svg|svgz|swf|tar|tif|tiff|ttf|ttc|_ttf|wav|wma|wri|woff|woff2|xla|xls|xlsx|xlt|xlw|zip|ASF|ASX|WAX|WMV|WMX|AVI|BMP|CLASS|DIVX|DOC|DOCX|EOT|EXE|GIF|GZ|GZIP|ICO|JPG|JPEG|JPE|WEBP|JSON|MDB|MID|MIDI|MOV|QT|MP3|M4A|MP4|M4V|MPEG|MPG|MPE|MPP|OTF|_OTF|ODB|ODC|ODF|ODG|ODP|ODS|ODT|OGG|PDF|PNG|POT|PPS|PPT|PPTX|RA|RAM|SVG|SVGZ|SWF|TAR|TIF|TIFF|TTF|TTC|_TTF|WAV|WMA|WRI|WOFF|WOFF2|XLA|XLS|XLSX|XLT|XLW|ZIP)$">
    FileETag MTime Size
    <IfModule mod_headers.c>
        Header set Pragma "public"
        Header append Cache-Control "public"
         Header unset Set-Cookie
         Header set X-Powered-By "W3 Total Cache/0.9.7"
    </IfModule>
</FilesMatch>
<FilesMatch "\.(bmp|class|doc|docx|eot|exe|ico|webp|json|mdb|mpp|otf|_otf|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|pot|pps|ppt|pptx|svg|svgz|swf|tif|tiff|ttf|ttc|_ttf|wav|wri|woff|woff2|xla|xls|xlsx|xlt|xlw|BMP|CLASS|DOC|DOCX|EOT|EXE|ICO|WEBP|JSON|MDB|MPP|OTF|_OTF|ODB|ODC|ODF|ODG|ODP|ODS|ODT|OGG|PDF|POT|PPS|PPT|PPTX|SVG|SVGZ|SWF|TIF|TIFF|TTF|TTC|_TTF|WAV|WRI|WOFF|WOFF2|XLA|XLS|XLSX|XLT|XLW)$">
    <IfModule mod_headers.c>
         Header unset Last-Modified
    </IfModule>
</FilesMatch>
<IfModule mod_headers.c>
    Header set Referrer-Policy ""
</IfModule>
# END W3TC Browser Cache
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress


# Wordfence WAF
<Files ".user.ini">
<IfModule mod_authz_core.c>
    Require all denied
</IfModule>
<IfModule !mod_authz_core.c>
    Order deny,allow
    Deny from all
</IfModule>
</Files>

# END Wordfence WAF

Does it mean: "we registered a 503 Error" or "we are intentionally causing a 503 Error for the following..."

And if so, how do I change it, and will it resolve the problem?

When I type in: https://example.de/robots.txt it also displays it, but when I check it with testing tools (google robots.txt Checker or Merkle) it does not show up.

wordpress.htaccessseo


Solution 1:[1]

When Googlebot visits you set an environment variable named bots to the value 1:

SetEnvIfNoCase User-Agent "Googlebot" bots

That looks at the User-Agent header, and if it contains "Googlebot" sets the bots variable to the default value (which is 1).

Then later you intentionally serve a 503 Service Unavailable status when the bots variable is 1:

RewriteEngine on
RewriteCond %{ENV:bots} 1
RewriteRule .* - [R=503,L]

You are also including a header to tell the bots to come back in 12 hours:

#### Retry 12h
Header always set Retry-After "43200" env=bots

This makes me think that rule was put in place as a temporary measure to handle some high traffic event, and then never removed.

To fix this, you should remove this code from your .htaccess file and allow bots to crawl your site. At the very least you should remove googlebot and bingbot from your disallow list because those are search engine spiders. When they can't crawl, it completely kills your SEO.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Stephen Ostermiller

Related Questions

Modify WooCommerce REST API product image upload

Modify WooCommerce REST API product image upload

Briefly unavailable for scheduled maintenance. Check back in a minute. No .maintenance File

Adding the custom page with add_menu_page function on Wordpress

How to get rid of ampersand using WP Nonce URL and WP Redirect or PHP header

How to I bring my sticky navbar in front of the contents on my page?

Invalid value in field "itemtype" in Google Search Console

Wordpress widget how to only display sub-categories based on selected Parent Category?

Custom Button next to “ADD TO CART” button of WooCommerce based on Product Type

Fatal error: Maximum execution time of 30 seconds exceeded in C:\xampp\htdocs\AKS_Shop\wp-includes\Requests\Transport\cURL.php on line 463

How to add automatically collapse/expand in content wordpress (single.php)?

woocommerce change position of price

Wordpress add responsive srcset header image to theme

Adding Multiple Custom Post Types in Wordpress

Submitting a form with ajax in Wordpress