Help SOE track the Lag (instructions inside)

[infinite loop]

Very good points Marsman, I suppose I was a bit quick to put all of it on SOE's lap. I'm just frustrated that their attitude seems to be that the problem is on our end, and they more or less deny the possibility of it being their problem. If they want us to report our ISP's and such, that's easy, and if it helps, we should do it.

Based on my play experience with the lag, I have come to my guess that the problem lies on their end. First of all, it did not start happening after the server merge. The first time I noticed it was on New Years' weekend. Secondly, it didn't start happening after any patch was released. It was well before the patch to remove the holiday stuff. And lastly, my outfit uses TS, and I've made an effort to ask across TS everytime I lag, if anyone else did. Every single time I have, every other person on TS responded with a yes. And my outfit is comprised of players from all across North America and different ISPs. I'm not debating you Marsman, I guess if it takes more badgering from us to get their network techs to look harder on their end, it's worth it. Deep down though, I know we're wasting our time.

[Phaden]

Thanks Marsman. But i can understand SOE trying to try the cheap solutions first. Marsman's example is so true, as any IT guy can account for. Not always the same example, but the same idea, try the easy stuff first before you commit resources.

[Marsman]

Well I agree also that there seems to be a problem and I concur with you that it's not the merge. One: as you you pointed out, it really didn't start after the merge but around the time of the holiday patches, and two - the merge doesn't explain werner which people report is experiencing the same symptoms of lag since the holiday patches as well - since they had no merge it really can't be atributed to that.
The TS correlation is exactly the type of thing that SOE needs to make sure their efforts are concentrated on the problem - keep the reports comming - and I am certain the dev team will take notice of it. They don't want an issue like this to ruin the game any more than we do - it's counterproductive to the entire dev team's efforts to have severe lag turn people away from the game. Hopefully Spork will give us an update soon - at least that they are continuing to investigate the problem.
One thread I did note pointed out that lag is usually minimal when you first start the game and progressively gets worse - plus when in an area where you are experiencing extreme lag, restarting the computer and going right back to the same area greatly lessens or completly remove the lag at that time. I have also noticed this trend where the longer you are on - the worse the lag seems to affect you. This may have to due with filling up the windows virtual memory swap file (which clears on restart). Perhaps memory management has a problem rather than the network code. In any event I think we all agree something is different from around the time of the holiday patches and despite there were no network code changes - something is affecting lag.

[ZeroShadow]

Ok, i have a fps of 100 in sanc and 20-60 in a heavy battle. And a ping of 40-60
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\matt carr>tracrt 199.108.204.48
'tracrt' is not recognized as an internal or external command,
operable program or batch file.

C:\Documents and Settings\matt carr>tracrt199.108.204.48
'tracrt199.108.204.48' is not recognized as an internal or external command,
operable program or batch file.

C:\Documents and Settings/Zeroshadow>tracert 199.108.204.48

Tracing route to ablpls-01-16.planetside.com [199.108.204.48]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.0.1
2 8 ms 8 ms 24 ms 10.177.4.1
3 8 ms 9 ms 8 ms 12.244.112.65
4 9 ms 9 ms 10 ms 12.244.69.25
5 9 ms 9 ms 10 ms 12.244.69.17
6 10 ms 9 ms 12 ms 12.244.73.18
7 10 ms 11 ms 11 ms gbr6-p80.dlstx.ip.att.net [12.123.17.30]
8 12 ms 12 ms 13 ms tbr1-p012701.dlstx.ip.att.net [12.122.12.69]
9 28 ms 24 ms 26 ms tbr2-cl1.attga.ip.att.net [12.122.2.90]
10 24 ms 24 ms 24 ms gbr2-p20.attga.ip.att.net [12.122.12.38]
11 26 ms 27 ms 28 ms gar1-p370.attga.ip.att.net [12.123.21.5]
12 37 ms 38 ms 41 ms mdf1-gsr12-2-pos-7-0.wdc1.attens.net [12.122.255
.178]
13 40 ms 39 ms 41 ms mdf1-bi8k-1-eth-2-1.wdc1.attens.net [63.240.192.
238]
14 52 ms 41 ms 40 ms vl10.ashaens-1.sonyonline.net [63.240.201.58]
15 39 ms 39 ms 38 ms ablpls-01-16.planetside.com [199.108.204.48]

Trace complete.

[Marsman]

pings and latency don't seem to be the issue here. We all know we can turn on the latency hud in PS (I believe it's called the performance hud - Shift + Peroid ) which shows ping, packet loss, and fps. But there is another setting that I don't think there is a key binding for. You can enable this by typing /stats

You will notice an additional hud under the performace hud which shows Upload and Download bytes per second. These also toggle green, yellow, and red like the permance hud figures do for good, warning, and bad conditions.

What I find is that when lag is low, my Download b/s is green (<1000) but when lag is present, it seems to cycle between green and red often jumping to 4000 to 6000 or more - if the lag is really bad - it can stay red almost constantly at 4000+ and only occationally dipping into the green area.

It would seem the server is getting backlogged trying to send me information - often for many seconds (as people have noticed from reloads, doors opening, etc..). Has anyone else enabled this hud and noticed the download rate? Btw, my upload almost never turns red or yellow - and seems to stay relativly contant. I don't know if this has any direct relation to the lag problem or not but I'd be curious to see what other peoples stats show on this hud? One other thing, if I completely remove myself from the area that is getting bad lag - i.e. respawn at another base, jump and a gal and move away, my stats will return to normal levels and the lag goes away. Same if I restart the machine. Also lag seems to affect machines differently - some experience fps crashes, others get gameplay delays - some get both and even at times some people have neither althought lately that is far less common. For the most part, laggy for some almosts always means at least some lag to everyone - some just worse than others. Thoughts?

[Edited as per following post- lol - thanks]

[Unknown]

...Or just type /stats to get the stat hud . I leave my stat hud on all the time, and I notice the download stream increases as the amount of people around increase, it's not directly related to the lag for me (sure, when the number gets bigger I lag, but that's a given, when the number is relatively small I still lag in some instances.)

[Marsman]

Thank you I just learned that was the command for that.

[Unknown]

You're very welcome. It'd save people alot of time over the method you posted hehe.

[Marsman]

Originally Posted by Unknown

You're very welcome. It'd save people alot of time over the method you posted hehe.

Quite!

[Tssha]

I think this problem may be a population problem. Think about it, this all started on the New Year weekend. This is the time when most people return from visiting relatives and have access to their gaming machines again. It's also the time when they start to use some of their holiday gifts, and some people got the gift of Planetside, and a play card to start them off. I'm betting a lot of people got Planetside for christmas, and this would explain how people in Werner could have a problem with population when they didn't have a server merge.

Even though the downstream isn't big to us, they gotta use a pretty big pipe to send it to so many people. It might be a bandwidth problem (though they might have that covered, I don't know). Another possibility is that there may be a queue, as someone else proposed. That data may be held up because of the sheer volume of data being sent. When you move away from an area, the server removes you from that queue, and your data isn't held up by it anymore. You no longer have to be sent the data on everyone at that base or on that continent.

Also, virtual memory may contribute to this in some way, though I have no idea how. It may be one element among many. The more we find out about this, the sooner SOE can fix this. Keep those ideas coming, and maybe we'll come up with the answer as a whole.

If everyone keeps posting ideas, I'll try to summarize them, and maybe I'll see a connection where you don't. With enough time, we may lick this problem yet. After all, with all of us working together to solve this problem, our collective wits may come up with a solution to this perplexing problem. 1000 eyes searching are better than 2.

[infinite loop]

Marsman - after reading your idea about downstream, I decided to check it out. I always have the stat window open, and I've noticed I frequently get into the red during big fights. But I never looked at it when these lag spikes were happening. Last night, on at least 3 of the lag spikes, my downstream shot up to over 10,000B/s (10K), and at one point was hovering around 16K for a few seconds. You may be on to something with the idea that the server is either overloaded, or has some other problem that causes it to freeze, and then catch up by sending a bunch of data at once. Could be one thing is causing the other actually. All I know is, every time I get the "lag", my downstream gets really high.

But when you think about it, the sheer amount of downstream shouldn't be lagging our client machines, especially those with broadband. My cable connection gets me ~200Kb/s down, so 10-16Kb/s shouldn't make my machine even blink. Very strange.

[ens.uk]

World: Werner
My Region: South Britain
My ISP: Blueyonder
Time and Day of TraceRT: 21.26 GMT- 21/01/04
Tracrt:


Tracing route to amspls-01-16.planetside.com [195.33.135.48]
over a maximum of 30 hops:

1 7 ms 10 ms 12 ms 10.86.0.1
2 13 ms 8 ms 8 ms gsr01-cr.blueyonder.co.uk [62.30.112.33]
3 10 ms 11 ms 10 ms tele1-cro-pos.telewest.net [194.117.136.34]
4 11 ms 8 ms 12 ms eu-gw1.telewest.net [194.117.136.186]
5 17 ms 18 ms 33 ms amix2nap.am.nl.ibm.net [195.69.144.191]
6 20 ms 26 ms 20 ms nlrtrd1102cr2--0-1-2-0.rd.nl.ip.att.net [165.87.
211.69]
7 19 ms 26 ms 24 ms nlamtr1102cr2--0-1-3-0.am.nl.ip.att.net [165.87.
212.38]
8 21 ms 23 ms 24 ms nlamtr1101er1-15-0.am.nl.ip.att.net [165.87.217.
158]
9 27 ms 20 ms 20 ms vl50.amsixmsfc-1.sonyonline.net [195.33.129.76]

10 21 ms 21 ms 21 ms amspls-01-16.planetside.com [195.33.135.48]

Trace complete.

Additional Comments : Can some give me feedback on this route. Is it slow or anything like that ?

[Marsman]

tracert is Trace Route - it sends 3 pings (a ping measures rtt or round trip time) to each router (a device which controls or routes internet traffic) in the path starting from your location and proceeding a step at a time towards the final destination.
3 pings are used to get an average and to show consistency. Seeing small increments in time along each step is normal and the 3 pings should be close in value. Large changes in value are indications of problems or severe congestion. If a single router responds with 17ms 465ms 40ms - I'd believe that router is having intermittent problems. If the progression of average values jumps between steps (7,10,13,15,458,464,472 for example) then the 5th router in this example is having problems adding aprox 440ms to the latency. Most posts so far have show fairly consistent router pings and normal progressive values through the steps indication no problems along the path.

[Biff]

Hrmm.. when I do shift+. or /stats I do not see the upstream, downstream bars as you guys describe. Is there something else I need to do?

I do get the ping and fps box though...

[Marsman]

/stats is the upstream downstream hud - not sure why you aren't seeing it if you typed /stats in the chat window. Should appear right under the fps hud.