Skip to content

#95 Implement cache purging (admin + non-admin) with timing + logging #97

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vpstackhub wants to merge 8 commits into
base: dev
Choose a base branch
from
Open

#95 Implement cache purging (admin + non-admin) with timing + logging #97

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
91893a1
#95 – Implement cache purge fallback (admin + non-admin) + timing logs
vpstackhub Dec 5, 2025
47d2df6
Remove hard-coded project.dir from project.properties (#95)
vpstackhub Dec 8, 2025
ce99e1e
Improve soft purge exception logging (Issue #95)
vpstackhub Dec 8, 2025
ebd641d
Add logging for soft purge attempts (Issue #95)
vpstackhub Dec 8, 2025
e679860
#95 - Replace purge method strings with enum, fix UI panel, update CLI
vpstackhub Dec 8, 2025
a1d9ed1
Merge branch 'dev' into feature/cache-purge-95
jamesmarkchan Dec 28, 2025
c02aca9
#95 minor house keeping
jamesmarkchan Dec 29, 2025
a4c3e32
#95 allow project to build
jamesmarkchan Dec 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions build.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@
<!-- version properties -->
<property name="pkg.name" value="jdiskmark"/>
<property name="app.name" value="JDiskMark"/>
<property name="version" value="0.6.3-dev.fat"/>
<property name="version" value="0.6.3-purge"/>
<!-- full semver does not work for msi product versions -->
<property name="msi.version" value="0.6.3"/>
<property name="msi.app.title" value="${app.name} ${version}"/>
Expand Down Expand Up @@ -114,11 +114,8 @@
<copy file="EmptyStandbyList.exe" todir="dist"/>
<copy file="capacity.ps1" todir="dist"/>
<copy file="disk-model.ps1" todir="dist"/>
<!--<antcall target="buildinfo"/>-->
<copy file="pkg/macos/JDM.icns" todir="${dist.dir}"/>
<copy file="pkg/macos/run.sh" todir="${dist.dir}"/>
<!--<copy file="build.properties" todir="dist"/>-->
<!--<copy file="build.properties" todir="src/META-INF"/>-->
<copy file="README.md" todir="dist"/>
<mkdir dir="tmp"/>
<mkdir dir="tmp/${release.dir}"/>
Expand Down
2 changes: 1 addition & 1 deletion nbproject/project.properties
View file
Open in desktop
Copy link
Member

@jamesmarkchan jamesmarkchan Dec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vpstackhub what version of netbeans are you using, interesting that this got checked in which seems to break running in netbeans for others?

image

specifically this line because it does not exist for other users.

project.dir=C:/Users/valer/OneDrive/Documenti/NetBeansProjects/jdm-java

Might you know why these attributes are being added into the project file?

Original file line number Diff line number Diff line change
Expand Up @@ -116,4 +116,4 @@ run.test.modulepath=\
${javac.test.modulepath}
source.encoding=UTF-8
src.dir=src
test.src.dir=test
test.src.dir=test
70 changes: 54 additions & 16 deletions src/jdiskmark/App.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,21 @@ public enum Mode { CLI, GUI }
// benchmarks and operations
public static HashMap<String, Benchmark> benchmarks = new HashMap<>();
public static HashMap<String, BenchmarkOperation> operations = new HashMap<>();
//Cli Mode
public static boolean isCliMode = false;

public static boolean isWindows() {
return os != null && os.toLowerCase().contains("win");
}

public static boolean isLinux() {
return os != null && os.toLowerCase().contains("linux");
}

public static boolean isMac() {
return os != null && os.toLowerCase().contains("mac");
}

/**
* @param args the command line arguments
*/
Expand Down Expand Up @@ -160,29 +174,59 @@ public static String getVersion() {
/**
* Initialize the GUI Application.
*/
/**
* Initialize the GUI Application.
*/
Copy link
Member

@jamesmarkchan jamesmarkchan Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is super minor but lines 173-174 lost a valid space.

public static void init() {


// ========= SAFE ELEVATION CHECK (NETBEANS + EXE/JAR) ============
if (!UtilOs.isProcessElevated()) {

// Detect if running inside NetBeans (dev mode)
String classPath = System.getProperty("java.class.path");
boolean runningInNetBeans =
classPath != null && classPath.toLowerCase().contains("netbeans");

if (!runningInNetBeans) {
// Only elevate when running as packaged JAR/EXE
System.out.println("Restarting JDiskMark with Administrator privileges...");
UtilOs.restartAsAdmin();
return;
} else {
System.out.println("Running inside NetBeans — skipping elevation.");
}
}
// ================================================================


os = System.getProperty("os.name");
arch = System.getProperty("os.arch");
processorName = Util.getProcessorName();
jdk = Util.getJvmInfo();


// Processor name based on OS
if (os.startsWith("Windows")) {
processorName = UtilOs.getProcessorNameWindows();
} else if (os.startsWith("Mac OS")) {
processorName = UtilOs.getProcessorNameMacOS();
} else if (os.contains("Linux")) {
processorName = UtilOs.getProcessorNameLinux();
}

checkPermission();
if (!APP_CACHE_DIR.exists()) {
APP_CACHE_DIR.mkdirs();
}

if (mode == Mode.GUI) {
loadConfig();
}

// initialize data dir if necessary
if (locationDir == null) {
locationDir = new File(System.getProperty("user.home"));
dataDir = new File(locationDir.getAbsolutePath()
+ File.separator + DATADIRNAME);
}

if (mode == Mode.GUI) {
Gui.configureLaf();
Gui.mainFrame = new MainFrame();
Expand All @@ -193,25 +237,19 @@ public static void init() {
Gui.mainFrame.setLocationRelativeTo(null);
Gui.progressBar = Gui.mainFrame.getProgressBar();
}

if (App.autoSave) {
// configure the embedded DB in .jdm
System.setProperty("derby.system.home", APP_CACHE_DIR_NAME);
loadBenchmarks();
}

if (mode == Mode.GUI) {
// load current drive
Gui.updateDiskInfo();
Gui.mainFrame.setVisible(true);
// save configuration on exit...
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() { App.saveConfig(); }
});
Runtime.getRuntime().addShutdownHook(new Thread(App::saveConfig));
}
}

public static void checkPermission() {
String osName = System.getProperty("os.name");
if (osName.contains("Linux")) {
Expand Down
69 changes: 62 additions & 7 deletions src/jdiskmark/Benchmark.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

package jdiskmark;

import com.fasterxml.jackson.annotation.JsonIgnore;
Expand Down Expand Up @@ -27,6 +26,8 @@
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.List;
import jakarta.persistence.Enumerated;
import jakarta.persistence.EnumType;
import java.util.UUID;

/**
Expand Down Expand Up @@ -184,6 +185,37 @@ public void serialize(UUID value, JsonGenerator gen, SerializerProvider serializ
BenchmarkType benchmarkType;
public BenchmarkType getBenchmarkType() { return benchmarkType; }

public enum CachePurgeMethod {
Copy link
Member

@jamesmarkchan jamesmarkchan Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vpstackhub I was thinking how about calling the top level operation CacheResetMethod this way if we know we need to do a cache reset based on the user being admin or not we can either DROP (admin removal) or PURGE (cache overwrite). This way the parent operation is called something different from the actual mechanism used.

public enum CacheClearanceMethod {
    NONE,
    DROP,  // Requires Admin
    PURGE  // Fallback for Non-Admin
}

alternative could use CacheClearanceMethod or CacheMitigationMethod.

NONE,
DROP_CACHE,
SOFT_PURGE;

@Override
public String toString() {
return switch (this) {
case DROP_CACHE -> "Drop Cache (OS Flush)";
case SOFT_PURGE -> "Soft Purge (Read-Through)";
case NONE -> "None";
};
}
}

// ---------------------------------------------------
// Cache purge metadata (for read-after-write benchmarks)
// ---------------------------------------------------
@Column
boolean cachePurgePerformed;

@Column
long cachePurgeSizeBytes;

@Column
long cachePurgeDurationMs;

@Column
@Enumerated(EnumType.STRING)
CachePurgeMethod cachePurgeMethod;

// timestamps
@Convert(converter = LocalDateTimeAttributeConverter.class)
@Column(name = "startTime", columnDefinition = "TIMESTAMP")
Expand All @@ -194,7 +226,7 @@ public void serialize(UUID value, JsonGenerator gen, SerializerProvider serializ


@OneToMany(mappedBy = "benchmark", cascade = CascadeType.ALL, orphanRemoval = true)
List<BenchmarkOperation> operations;
List<BenchmarkOperation> operations = new ArrayList<>();

public List<BenchmarkOperation> getOperations() {
return operations;
Expand Down Expand Up @@ -252,16 +284,22 @@ public String toResultString() {
}

public Benchmark() {
operations = new ArrayList<>();
startTime = LocalDateTime.now();
cachePurgePerformed = false;
cachePurgeSizeBytes = 0L;
cachePurgeDurationMs = 0L;
cachePurgeMethod = CachePurgeMethod.NONE;
appVersion = App.VERSION;
profileName = App.activeProfile.getName();
}

Benchmark(BenchmarkType type) {
operations = new ArrayList<>();
startTime = LocalDateTime.now();
benchmarkType = type;
cachePurgePerformed = false;
cachePurgeSizeBytes = 0L;
cachePurgeDurationMs = 0L;
cachePurgeMethod = CachePurgeMethod.NONE;
Copy link
Member

@jamesmarkchan jamesmarkchan Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If interested lines 200-203 and 209-212 can be set at declaration I think since the values are constants.

image

appVersion = App.VERSION;
profileName = App.activeProfile.getName();
}
Expand Down Expand Up @@ -295,7 +333,24 @@ public String getDuration() {
long diffMs = Duration.between(startTime, endTime).toMillis();
return String.valueOf(diffMs);
}


// cache purge getters (for UI / JSON)
public boolean isCachePurgePerformed() {
return cachePurgePerformed;
}

public long getCachePurgeSizeBytes() {
return cachePurgeSizeBytes;
}

public long getCachePurgeDurationMs() {
return cachePurgeDurationMs;
}

public CachePurgeMethod getCachePurgeMethod() {
return cachePurgeMethod;
}

// utility methods for collection

@JsonIgnore
Expand Down Expand Up @@ -333,7 +388,7 @@ static int delete(List<UUID> benchmarkIds) {
.setParameter("benchmarkIds", benchmarkIds)
.executeUpdate();

// delete the parent BenchmarkOperation records
// delete the parent Benchmark records
String deleteBenchmarksJpql = "DELETE FROM Benchmark b WHERE b.id IN :benchmarkIds";
int deletedBenchmarksCount = em.createQuery(deleteBenchmarksJpql)
.setParameter("benchmarkIds", benchmarkIds)
Expand All @@ -347,4 +402,4 @@ static int delete(List<UUID> benchmarkIds) {
em.getTransaction().commit();
return deletedBenchmarksCount;
}
}
}
92 changes: 87 additions & 5 deletions src/jdiskmark/BenchmarkWorker.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ public static int[][] divideIntoRanges(int startIndex, int endIndex, int numThre
protected Benchmark doInBackground() throws Exception {

if (App.verbose) {
msg(">>> USING CUSTOM PURGE BUILD (V1)");
msg("*** starting new worker thread");
msg("Running readTest " + App.isReadEnabled() + " writeTest " + App.isWriteEnabled());
msg("num samples: " + App.numOfSamples + ", num blks: " + App.numOfBlocks
Expand Down Expand Up @@ -238,13 +239,77 @@ protected Benchmark doInBackground() throws Exception {
App.wIops = wOperation.iops;
Gui.mainFrame.refreshWriteMetrics();
}

System.out.println(">>> BenchmarkWorker: ENTERING CACHE PURGE SECTION");
Copy link
Member

@jamesmarkchan jamesmarkchan Dec 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this could be used: ENTERING CACHE RESET SECTION or ENTERING CACHE MITIGATION SECTION


// ------------------------------------------------------------
// Cache Purge Step (only for Read-after-Write scenario)
// ------------------------------------------------------------
if (App.isWriteEnabled() && App.isReadEnabled() && !isCancelled()) {

long purgeStartNs = System.nanoTime();
benchmark.cachePurgePerformed = true;
benchmark.cachePurgeMethod = App.isAdmin
Copy link
Member

@jamesmarkchan jamesmarkchan Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

potentially cachePurgePerformed > cacheResetPerformed

and cachePurgeMethod > cacheResetMethod

? Benchmark.CachePurgeMethod.DROP_CACHE
: Benchmark.CachePurgeMethod.SOFT_PURGE;

// Detect purge size roughly equal to the full test data footprint
long estimatedBytes = (long) App.numOfSamples * (long) App.numOfBlocks *
((long) App.blockSizeKb * 1024L);
Copy link
Member

@jamesmarkchan jamesmarkchan Dec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing that we might run into is if the bytes we purge is less than the actual disk cache the benchmark would still be affected by the disk cache. When i asked gemini it recommended purging the size of the entire main memory to be safe as that was the only way to guarantee that the disk cache does not affect our benchmark even if the disk cache is usually just a portion of main memory and for this reason if we go the safe route the the process might be a little lengthy since different system could have different amounts of memory installed, might be worth notifying user what is it doing so they understand why the clearing is taking so long. maybe we can default to purging the size of system memory + 10% (need to detect it from ), i think it's simply Runtime.getRuntime().totalMemory(); and maybe later can add an advanced option for the user to do the cache purging based on a percentage of the main memory if we find safe values for this. @jslcom are you familiar with the default disk cache sizes?

benchmark.cachePurgeSizeBytes = estimatedBytes;
Copy link
Member

@jamesmarkchan jamesmarkchan Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is set from zero to the actual purge size and only if not admin and a purge operation occurs.


Gui.msg("⚡ Starting cache purge (" + benchmark.cachePurgeMethod +
", ~" + (estimatedBytes / (1024*1024)) + " MB)");

if (App.isAdmin) {
// ==========================
// ADMIN MODE: Use drop-cache
// ==========================

System.out.println(">>> BenchmarkWorker: ADMIN purge path selected");
System.out.println(">>> BenchmarkWorker: Calling UtilOs.dropOsCache() ...");

try {
boolean ok = UtilOs.dropOsCache();

System.out.println(">>> BenchmarkWorker: dropOsCache() returned = " + ok);

if (!ok) {
System.out.println(">>> BenchmarkWorker: dropOsCache FAILED — fallback to SOFT purge");
Gui.msg("⚠ drop-cache failed — switching to soft purge.");

System.out.println(">>> BenchmarkWorker: Starting SOFT purge (fallback)");
Util.readPurge(estimatedBytes, pct -> setProgress(Math.min(99, pct)));
System.out.println(">>> BenchmarkWorker: Soft purge completed.");
}
} catch (Exception ex) {
System.out.println(">>> BenchmarkWorker: EXCEPTION in dropOsCache — fallback to read purge");
ex.printStackTrace();
Gui.msg("⚠ Exception during drop-cache, fallback to read purge.");
Util.readPurge(estimatedBytes, pct -> setProgress(Math.min(99, pct)));
}

// TODO: review renaming all files to clear catch
if (App.isReadEnabled() && App.isWriteEnabled() && !isCancelled()) {
// TODO: review refactor to App.dropCache() & Gui.dropCache()
Gui.dropCache();
} else {
// ==========================
// USER MODE: Soft purge
// ==========================
System.out.println(">>> BenchmarkWorker: Starting SOFT purge (readPurge)");
Gui.msg("Performing soft cache purge (read-through)…");

Util.readPurge(estimatedBytes, pct -> setProgress(Math.min(99, pct)));

System.out.println(">>> BenchmarkWorker: Soft purge completed.");
}

long purgeEndNs = System.nanoTime();
long purgeDurationMs = (purgeEndNs - purgeStartNs) / 1_000_000L;

benchmark.cachePurgeDurationMs = purgeDurationMs;

Gui.msg("✔ Purge finished in " + purgeDurationMs + " ms");
}


if (App.isReadEnabled()) {
BenchmarkOperation rOperation = new BenchmarkOperation();
rOperation.setBenchmark(benchmark);
Expand Down Expand Up @@ -369,7 +434,11 @@ protected void process(List<Sample> sampleList) {
case Sample.Type.WRITE -> Gui.addWriteSample(s);
case Sample.Type.READ -> Gui.addReadSample(s);
}
});
});
if (App.isCliMode) {
int pct = this.getProgress();
printCliProgress(pct);
}
}

@Override
Expand All @@ -380,4 +449,17 @@ protected void done() {
App.state = App.State.IDLE_STATE;
Gui.mainFrame.adjustSensitivity();
}

private void printCliProgress(int pct) {
int bars = pct / 5; // 20-bar progress
Copy link
Member

@jamesmarkchan jamesmarkchan Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super minor, there is a java practice where the recommendation is to avoid abbreviations when possible so percent instead of pct.

int spaces = 20 - bars;

String bar = "[" + "#".repeat(bars) + "-".repeat(spaces) + "] " + pct + "%";

System.out.print("\r" + bar);

if (pct >= 100) {
System.out.println();
}
}
}
Loading