Time for Dedicated Server?

General Comments, Questions about all things OmnipotentS that don't go in other topics/forums
User avatar
Anonymous.
Posts: 371
Joined: Sat Jun 12, 2021 10:54 pm

Re: Time for Dedicated Server?

Post by Anonymous. »

pooty wrote: Tue Aug 23, 2022 8:28 am Good news is that the server move really seemed to help the CPU usage. 26 players and barely a tick above 40% CPU. So obviously there's some sort of bottleneck on the netspeed (its a 1 GB network on server), so I think its an engine issue, since there doesn't seem to be a hardware limitation.
CPU usage isn't useful beyond a basic idea of what the CPU is doing. But even then, you're going off total usage on a multicore CPU when UT is bound by a single thread, meaning one core is always pinned at 100%. Trust me, I've been dealing with single thread bottlenecks for a long time and the only fix is brute force with overclocked, high IPC CPUs. Server CPUs can't be overclocked (or even lock clocks), and unlocked clocks are retarded because load balancing schedulers always fuck around with what cores the threads run on (clock switching takes an eternity).

https://www.brendangregg.com/blog/2017- ... wrong.html
  • The problem with averages: 80% utilized over 1 minute, hiding bursts of 100%.
  • Turboboost varying the clockrate.
I don't know how much control you have over the OS, but is the power plan set to high performance? If hyperthreading is disabled, you can trick the CPU into locking its clocks by disabling idle, which is a hidden option in Windows power plan settings which keeps the cores in C0. With idle disabled you'll get much more consistent performance, especially with interrupt handling.

admin cmd.exe:
Disable idle on:

Code: Select all

powercfg -setacvalueindex scheme_current sub_processor 5d76a2ca-e8c0-402f-a133-2158492d58ad 1
powercfg -setactive scheme_current
Disable idle off (default):

Code: Select all

powercfg -setacvalueindex scheme_current sub_processor 5d76a2ca-e8c0-402f-a133-2158492d58ad 0
powercfg -setactive scheme_current
Another place to look is in the network adapter's settings, namely interrupt moderation and RSS. Interrupt moderation off is ideal for lowest network latency, but comes at the cost of DoSing the CPU with interrupts. With IMod enabled you can change the interrupt moderation rate; "minimal" will yield better performance stability than if interrupt moderation were turned off. There are more aggressive options, but I'm not sure how it would impact a game server where precise timing is required. RSS on is generally a good idea for servers if there is a lot of network activity since it puts more than one core to work.

Image
User avatar
captainsnarf
Posts: 2699
Joined: Tue Jul 06, 2021 1:51 pm
Location: Washington
Server Sponsor: Yes
Server Admin: Yes
Contact:

Re: Time for Dedicated Server?

Post by captainsnarf »

wow anon, you are quite knowledgeable about optimizing PC usage for games!

Image
User avatar
pooty
Posts: 4513
Joined: Sat Apr 03, 2021 10:22 am
Location: Michigan
Server Sponsor: Yes
Server Admin: Yes

Re: Time for Dedicated Server?

Post by pooty »

CPU usage isn't useful beyond a basic idea of what the CPU is doing.
So I asked NFO, they provide two metrics, one for our "server" usage, which is just our UT install and one for the overall server. The one I am referring to is just us, I specifically asked about the Multicore thing and they were aware of the single core/UT thing (they host many of the UT servers).

I've also asked about hyperthreading and been told its disabled.

I don't have any control over the OS, that's all them. I would hope they've done a bit of optimizing as they are a gaming hosting company.

I am happy to ask them about other optimizations, they've always been good about answering questions.

I think the other thing is to test it out and see if we have the same issues with all the optimizations.

Other than a few lags, I thought last night on Minus-KS was much better than before..
User avatar
Anonymous.
Posts: 371
Joined: Sat Jun 12, 2021 10:54 pm

Re: Time for Dedicated Server?

Post by Anonymous. »

captainsnarf wrote: Thu Aug 25, 2022 7:52 am wow anon, you are quite knowledgeable about optimizing PC usage for games!
Thanks, I owe most of my skill to it.
pooty wrote: Thu Aug 25, 2022 8:52 am So I asked NFO, they provide two metrics, one for our "server" usage, which is just our UT install and one for the overall server. The one I am referring to is just us, I specifically asked about the Multicore thing and they were aware of the single core/UT thing (they host many of the UT servers).

I've also asked about hyperthreading and been told its disabled.

I don't have any control over the OS, that's all them. I would hope they've done a bit of optimizing as they are a gaming hosting company.

I am happy to ask them about other optimizations, they've always been good about answering questions.

I think the other thing is to test it out and see if we have the same issues with all the optimizations.

Other than a few lags, I thought last night on Minus-KS was much better than before..
Oh ok sorry, I thought you were talking about total CPU usage because going off the processor having 8 cores, that would mean we are using nearly 3 cores at times (is that extra load just from network traffic?). Those optimizations I mentioned are a bit of a reach to ask from them since they will affect the other clients too, for better or worse. If the server is a VPS, that also doesn't help our case.

Unfortunately yesterday it was still a bit laggy - most noticeable when entering/exiting vehicles or switching seats. I had a few noreg streaks as well: https://streamable.com/p0ruin
User avatar
pooty
Posts: 4513
Joined: Sat Apr 03, 2021 10:22 am
Location: Michigan
Server Sponsor: Yes
Server Admin: Yes

Re: Time for Dedicated Server?

Post by pooty »

Unfortunately yesterday it was still a bit laggy
Yep. I think it might be worth renting a dedicated server for a month and testing it out. I'd kick in a few bucks, and I know several others that would.
Before we do that though, the other "variable" is we are running UTComp's enhanced netcode. IMO that's been more laggy than previously...but its a bit of subjective judgement, but I don't feel we had these lag issues prior to it.

Can we turn off the enhanced netcode? I think we can, might be worth a few day test without it. I definitely don't want to get rid of UTComp, I love the customized scoring and like the hitsounds. I know it helps with Ping compensation, but is that causing the lag?
User avatar
captainsnarf
Posts: 2699
Joined: Tue Jul 06, 2021 1:51 pm
Location: Washington
Server Sponsor: Yes
Server Admin: Yes
Contact:

Re: Time for Dedicated Server?

Post by captainsnarf »

With it turned on in the server, players have the option of turning it off individually when they hit F5.
If it's off in the server, then it is off for everybody and players cannot change it.

Honestly enhanced net code is the primary reason I want UTComp. The other features are OK but the net code is the killer feature for me. I'd rather you try lowering the update rate down from 200 before trying disabling it. Maybe try 150 or even 100.

Also, I'm ready to pony up the cash for a dedicated server if needed.
User avatar
pooty
Posts: 4513
Joined: Sat Apr 03, 2021 10:22 am
Location: Michigan
Server Sponsor: Yes
Server Admin: Yes

Re: Time for Dedicated Server?

Post by pooty »

Honestly enhanced net code is the primary reason I want UTComp. The other features are OK but the net code is the killer feature for me. I'd rather you try lowering the update rate down from 200 before trying disabling it. Maybe try 150 or even 100.
That's what I was looking for...we definitely want to keep the enhanced netcode.
For the curious:
https://www.pnw4runners.com/gaming/utco ... tm#topic41
https://beyondunrealwiki.github.io/page ... ation.html

I've set the NewNetUpdateFrequency=125, default for UT is 100. I do think 200 is overkill, 200 updates a second..most won't have framerates over 85, without increased netspeed, and most even then, for example, I typically see 115-150 range, so no point to having all those "extra" updates, IMO. I think this makes sense thinking about how the netspeed increase bogged the server (and perhaps if this works we can try 10001 netspeed again)--not sure how those interact.

BTW, the UI still lets you set it to whatever you want, but I think looking the code, it won't go over server setting (MutatorOwner).
User avatar
captainsnarf
Posts: 2699
Joined: Tue Jul 06, 2021 1:51 pm
Location: Washington
Server Sponsor: Yes
Server Admin: Yes
Contact:

Re: Time for Dedicated Server?

Post by captainsnarf »

Uhg, put it back please. It sucks at 125, a lot. Testing with bots I had so many unreg hits now.

I think NewNetUpdateFrequency needs to be >= 200 because the default for actor is 100, and it is doing a ping/pong.

*edit*

I tried again and did a little better. I still think it's not as good as it was. I guess lets let it go a night and see first.
User avatar
McLovin
Posts: 1193
Joined: Sat Apr 03, 2021 12:54 pm
Location: Salt Lake City, Utah
Server Sponsor: Yes
Server Admin: Yes

Re: Time for Dedicated Server?

Post by McLovin »

pooty wrote: Thu Aug 25, 2022 10:25 am ...
I think it might be worth renting a dedicated server for a month and testing it out. I'd kick in a few bucks, and I know several others that would.
...
I say do it. I'll kick in some if it makes things better.
User avatar
pooty
Posts: 4513
Joined: Sat Apr 03, 2021 10:22 am
Location: Michigan
Server Sponsor: Yes
Server Admin: Yes

Re: Time for Dedicated Server?

Post by pooty »

I think NewNetUpdateFrequency needs to be >= 200 because the default for actor is 100, and it is doing a ping/pong.
I is confused.
Actor is update 100Hz, which according to that page is the most frequent update, so I guess what exactly is the new value doing? Is it changing that value, changing all updates, or somehow a cap?
Post Reply