'java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled Java code

I was wondering if some of the JVM gurus out there can briefly explain the following error. What does it actually mean in technical terms and what are the sequences of events that can lead to this error?

java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled Java code
javamemoryjvm


Solution 1:[1]

This error may be induced by corrupted index which in turn becomes corrupted due to java.io.IOException: No space left on device

Look into your_kafka_home/logs/server.log

It all starts like this:

[2019-11-22 17:27:25,204] FATAL [Replica Manager on Broker 1]: Halting due to unrecoverable I/O error while handling produce request:  (kafka.server.ReplicaManager)
kafka.common.KafkaStorageException: I/O exception in append to log 'xxx-response-1-0'
    at kafka.log.Log.append(Log.scala:443)
    at kafka.cluster.Partition.$anonfun$appendRecordsToLeader$1(Partition.scala:451)
Caused by: java.io.IOException: No space left on device
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211)
    at org.apache.kafka.common.record.MemoryRecords.writeFullyTo(MemoryRecords.java:82)
    at org.apache.kafka.common.record.FileRecords.append(FileRecords.java:159)
    at kafka.log.LogSegment.append(LogSegment.scala:110)
    at kafka.log.Log.append(Log.scala:429)
    ... 21 more

is followed by (possibly mutilple times):

[2019-11-22 17:28:00,395] WARN Found a corrupted index file due to requirement
failed: Corrupt index found, index file (/opt/kafka/kafka-logs/__consumer_offsets- 
9/00000000000000000000.index) has non-zero size but the last offset is 0 which is no
larger than the base offset 0.}. deleting /opt/kafka/kafka-logs/__consumer_offsets-9
/00000000000000000000.timeindex, /opt/kafka/kafka-logs/__consumer_offsets-9
/00000000000000000000.index and rebuilding index... (kafka.log.Log)

and finally comes to :

[2019-11-22 17:28:02,025] FATAL [Kafka Server 1], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.InternalError: **a fault occurred in a recent unsafe memory access operation in compiled Java code**
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
    at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:213)
    at kafka.log.TimeIndex.maybeAppend(TimeIndex.scala:107)
    at kafka.log.LogSegment.$anonfun$recover$1(LogSegment.scala:238

Solution 2:[2]

This can also happen due to disk space issues.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1
Solution 2 z atef

Related Questions