Virtual threads + semaphore in front of BCrypt gave me +28% throughput, but the container is still pinned at its CPU limit. What to do next to optimize it?

[u/No_Expression_5450]

I'm load testing an auth service (Spring Boot, Java, Redis, PostgreSQL, Docker on a t3.large, k6). I replaced a dedicated 2-thread BCrypt pool (20-slot queue) with virtual threads plus a 6-permit semaphore.

Results at the same 40 RPS target (queue → virtual threads + semaphore):

• Throughput: 20.3 → 26.06 OPS (+28%). OPS = operations per second, the full k6 iteration mix.
• Login p95: 28.5s → 20.1s
• Timeouts: 43 → 0

That's still only ~65% of target, and a 20s login is unusable.

At 50 RPS throughput plateaued at ~28 OPS. docker stats showed the auth container averaging ~145% CPU (peak ~153%). My compose file sets a 1.5 CPU limit. Endpoints that were ~85ms p95 at 40 RPS slid to ~2s.

My read: virtual threads help with blocking I/O, not CPU-bound work. I think the gain came from removing the fast-reject queue rather than from adding capacity, but I haven't isolated that.

Caveats: 41% of my test traffic is logins (unrealistic), and I haven't measured a single-hash time yet.

The puzzle: BCrypt strength is 4 in my load-test config, which I'd expect to cost ~1-2ms per hash. At ~10 logins/sec, that shouldn't pin 1.5 cores. Suspects: the cost factor baked into the pre-seeded hashes (matches() uses the stored cost), JWT signing, GC, or my executor/semaphore wiring.

What would you profile first: async-profiler, JFR, thread dumps?

[u/AutoModerator]

Please ensure that:

• Your code is properly formatted as code block - see the sidebar (About on mobile) for instructions
• You include any and all error messages in full
• You ask clear questions

• You demonstrate effort in solving your question/problem - plain posting your assignments is forbidden (and such posts will be removed) as is asking for or giving solutions.

  Trying to solve problems on your own is a very important skill. Also, see Learn to help yourself in the sidebar

If any of the above points is not met, your post can and will be removed without further warning.

Code is to be formatted as code block (old reddit: empty line before the code, each code line indented by 4 spaces, new reddit: https://i.imgur.com/EJ7tqek.png) or linked via an external code hoster, like pastebin.com, github gist, github, bitbucket, gitlab, etc.

Please, do not use triple backticks (```) as they will only render properly on new reddit, not on old reddit.

Code blocks look like this:

public class HelloWorld {

    public static void main(String[] args) {
        System.out.println("Hello World!");
    }
}

You do not need to repost unless your post has been removed by a moderator. Just use the edit function of reddit to make sure your post complies with the above.

If your post has remained in violation of these rules for a prolonged period of time (at least an hour), a moderator may remove it at their discretion. In this case, they will comment with an explanation on why it has been removed, and you will be required to resubmit the entire post following the proper procedures.

To potential helpers

Please, do not help if any of the above points are not met, rather report the post. We are trying to improve the quality of posts here. In helping people who can't be bothered to comply with the above points, you are doing the community a disservice.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/GuyWithLag]

Yes, I know I'm talking to a bot.

Endpoints that were ~85ms p95 at 40 RPS slid to ~2s.

That doesn't make sense; Looks like you have a random assortment of BCrypt invocations and some of them are taking wayyyyy longer than others; with virtual threads you force-serialized these, so your P95 increased.

Look into what's taking so long in these cases; at 20 seconds it feels like there's some sort of timeout, check pool time / setup / utilization, and pepper timing measurements around.

[u/No_Expression_5450 → u/GuyWithLag]

thanks i will try it

[u/disposepriority]

I assume the "java help" you need is in developing a java application that cures you of the rare disease forcing you to post AI garbage on reddit?

[u/No_Expression_5450 → u/disposepriority]

If you think tearing down someone’s work is a way of being helpful, you should probably reconsider what you think ‘help’ is. If you don't want to help just scroll away

[u/GuyWithLag → u/No_Expression_5450]

Nah he has a point. We dont owe you anything.

[u/Working_Anything_759]

Check the cost factor in your pre-seeded database rows. BCrypt matches() reads the strength from the stored hash, so if those were generated with default settings or a higher number than 4, you are running expensive CPU-bound operations that virtual threads just queue up one by one. Profile the actual duration of a single match() call against one of those specific user records before touching any more executors

[u/Electrical_Being_813]

Virtual threads do not magically improve cpu usage. Try checking what is causing biggest gc load, and use object pooling for objects that are needed.

[u/jonathaz]

There is no such thing as .5 core. Unless you are pinned, limits are enforced by arbitrary CPU throttling. The throttling by the container will not be evenly distributed and your p95 will be much higher than the average. You can tune the JVM threading to do better. The first step is to understand your application performance in the context of the JVM. Virtual threads run on host threads, there is a pool of those. If those are the what’s using your CPU, that’s your 1st problem. Unknown pool size could be more or less than your semaphore of 6, which is larger than 1.5. Under load, the container will throttle it by randomly pausing threads. If you want your code to run at full speed, and you’re CPU bound, you need the CPU limit >= the number of worker threads. Virtual threads can give you some flexibility if you are IO bound, have a mix of CPU and IO, or if you are CPU bound and are OK with less than full speed. Java will schedule virtual threads more efficiently and fairly than the container throttling. You’d still need your CPU limit and your virtual thread worker pool set appropriately, let’s say 2 each. With your semaphore at 6, you would expect to run about 3x slower under load, but more consistently so. You could also be churning objects, memory bound, and straining the garbage collector. Or something else.