Jump to content

Let's talk about the BE kicks


bamf

Recommended Posts

1 hour ago, bamf said:

Sure.  The new change requiring you to "Show Storage" in your houses has had a very positive impact on overall memory performance.  Over the course of this past weekend, the number of BE mass kicks was down significantly.  Server 4 in particular has been much more stable since the update.  

As I've said all along, one mass kick is too many for me - so we will continue to do things to try to bring overall memory utilization down.  We have a few other things in mind that should be completely on the back end, but could be enormously successful in bringing down what the server executable stores in memory.  Also, we are hopeful that the 64-bit server application will alleviate the need to do more as well, but I'm sure it will have additional issues that we will need to resolve.  

Going by what you said, if you were to remove NPC's would that have the same affect as the storage crates being removed?

Because if that was the case, you could try remove the DP mission NPC's as for the most part there really buggy (on tanoa atleast) and they aren't a very popular way of making money

Edited by Eazy
Link to comment
3 minutes ago, Eazy said:

Going by what you said, if you were to remove NPC's would that have the same affect as the storage crates being removed?

Because if that was the case, you could try remove the DP mission NPC's as for the most part there really buggy (on tanoa atleast) and they aren't a very popular way of making money

+1, 

If we can remove some of the major things that no one really does, it could help.

I think i see one truck a month doing a DP mission. 

Link to comment
2 minutes ago, Dork said:

+1, 

If we can remove some of the major things that no one really does, it could help.

I think i see one truck a month doing a DP mission. 

Feature != Serverside memory being used.

Most NPCs are just a place for you to initialize all clientside code. Think of them as the key to your computer's "car"(the mission file).

Link to comment

4 mass kicks on S3 in less than 10 minutes.

During the summer people had no problem switching arma versions to play on altis or tanoa. So why not try to change them to run 64 bit in the development build instead of waiting for Bohemia to release a full fix. If it doesn't work after a couple of days of testing then switch back

Link to comment
12 hours ago, signal said:

4 mass kicks on S3 in less than 10 minutes.

During the summer people had no problem switching arma versions to play on altis or tanoa. So why not try to change them to run 64 bit in the development build instead of waiting for Bohemia to release a full fix. If it doesn't work after a couple of days of testing then switch back

Once a server throws that initial mass kick due to memory being too high, then it is mostly doomed until a hard restart. You're already at the limit you see, and players being kicked only frees a portion of the memory they made the server use during their play session (thus the memory leak). 

When a mass kick happens, the best thing you can do is let an admin know right away and hopefully they can get into the server to start a hard reset timer. 

Nightfury likes this
Link to comment
12 hours ago, signal said:

4 mass kicks on S3 in less than 10 minutes.

During the summer people had no problem switching arma versions to play on altis or tanoa. So why not try to change them to run 64 bit in the development build instead of waiting for Bohemia to release a full fix. If it doesn't work after a couple of days of testing then switch back

Over the summer you couldn't do a single drug runner mission without mass kicks, hence why everyone went to olympus for a bit (me included)

Link to comment
19 minutes ago, Sean. said:

Over the summer you couldn't do a single drug runner mission without mass kicks, hence why everyone went to olympus for a bit (me included)

Everyone is a bit of a huge over exaggeration. There was literally a few handful of people that went, and it wasn't all that much better.  Instead of getting kicked you just got what they called a "texture bug" which was in fact memory leaks, but they tried to make it sound like it wasn't.

Link to comment
2 minutes ago, HotWings said:

Everyone is a bit of a huge over exaggeration. There was literally a few handful of people that went, and it wasn't all that much better.  Instead of getting kicked you just got what they called a "texture bug" which was in fact memory leaks, but they tried to make it sound like it wasn't.

Because It's been a while since I've posted an updated version of this:

 

Asylum Unique Users over the past 2 years.

Qkf2ogAAAABJRU5ErkJggg==

Link to comment
12 minutes ago, HotWings said:

Everyone is a bit of a huge over exaggeration. There was literally a few handful of people that went, and it wasn't all that much better.  Instead of getting kicked you just got what they called a "texture bug" which was in fact memory leaks, but they tried to make it sound like it wasn't.

Its not really a huge exaggeration. Elysium, Insanity, Nv, Tenacity. Most players from the largest gangs all went over the play on Olympus for a short while, the rest who didn't just took a break. The Memory leak on Olympus was only for a few clients caused by looking at Police textures, but no one fought cops because there was Cartels all the time.

There was a very rough time in Asylum where the servers where literally unplayable above 70 pop, Gang Wars was a prime example of that.

 

11 minutes ago, Gnashes said:

 

Asylum Unique Users over the past 2 years.

You've heard this a lot in the past, numbers from Unique Users means fuck all. If Asylum didn't have such a rough couple of months, it would of had the player base to keep 6 Altis life servers up comfortable. New players don't stick around when they can't finish 1 run without being kicked.

All this graph shows us in the last year is that Unique ID's increased after Free Weekend and Sales, and increased again at Christmas and other Holidays which is normal. Probably not possible, but if you showed us a graph of average play time or average server pop through out the years, that would mean a lot more and gives us a better understanding of how well the servers are doing compared to previous months.

Link to comment
16 minutes ago, Haych said:

Its not really a huge exaggeration. Elysium, Insanity, Nv, Tenacity. Most players from the largest gangs all went over the play on Olympus for a short while, the rest who didn't just took a break. The Memory leak on Olympus was only for a few clients caused by looking at Police textures, but no one fought cops because there was Cartels all the time.

 

It is a huge exaggeration.  The servers were still populated without the gangs.  Gangs are not the only people that play on Asylum lol

Don and Bikstok like this
Link to comment
25 minutes ago, Haych said:

graph of average play time

I can take a shot at "actual playtime", using the average gap between "number of syncs" as a basis. Have to exclude some oddities when Paratus would do things and push one sync from every row of the DB to Splunk though.

Report is running. I wonder if I can cap out disk usage going through some odd 80,000,000 events.

Link to comment
34 minutes ago, HotWings said:

It is a huge exaggeration.  The servers were still populated without the gangs.  Gangs are not the only people that play on Asylum lol

Gangs and the APD make up the majority of the active community. Less gangs playing = Less APD logging in. Not really a huge exaggeration when the majority of active rebels either took a break, or started playing on another server.

What you have left is a semi populated server filled with hobos who probably don't even know what the forums are and the odd dedicated cops/people trying to get playtime. Even if we all decided to log on, as soon as the servers hit 70+ it was gg. 

Squirtle likes this
Link to comment
2 hours ago, Haych said:

What you have left is a semi populated server filled with hobos who probably don't even know what the forums are and the odd dedicated cops/people trying to get playtime. Even if we all decided to log on, as soon as the servers hit 70+ it was gg. 

If you look over the last 30 days though, we average about 449 players per hour connected to the servers (24/7), and while there are times of day that the servers are not full (obviously) there are equal amounts of time that you have to spam to get in.  

Also, let's look at the going forward (or near past) instead of trying to rehash months ago.  We've made tremendous progress in the last week in particular with working on what we believe is the root cause of the issue - server memory consumption.  When the admin team is able to stay on top of the memory and proactively hard restart servers with near critical memory utilization, then the number of kicks is reduced to nearly zero.  

HotWings, Squirtle, BSquare and 2 others like this
Link to comment
1 hour ago, bamf said:

If you look over the last 30 days though, we average about 449 players per hour connected to the servers (24/7), and while there are times of day that the servers are not full (obviously) there are equal amounts of time that you have to spam to get in.  

Also, let's look at the going forward (or near past) instead of trying to rehash months ago.  We've made tremendous progress in the last week in particular with working on what we believe is the root cause of the issue - server memory consumption.  When the admin team is able to stay on top of the memory and proactively hard restart servers with near critical memory utilization, then the number of kicks is reduced to nearly zero.  

Yes ok there are more playing coming in but, they are always leaving because of stupid "Mem- Leaks" Nothing will change till that is fixed, I never got issues like this in other big time Altis/Tanoa RP Servers. There is way to much unneeded stuff in the servers. We as a community would like to see how long new players stay and leave the servers. (Hours)

An other thing 

1 hour ago, bamf said:

server memory consumption

The server memory consumption is way too crazy. Including fights. There are way too many Issues with Fighting. Especially in Tanoa, don't blame the map for it, Its not a good excuse for the amount of mem-leaks and mass kicks. In general with Altis as well. 

Link to comment
1 minute ago, Chewbacca said:

Yes ok there are more playing coming in but, they are always leaving because of stupid "Mem- Leaks" Nothing will change till that is fixed, I never got issues like this in other big time Altis/Tanoa RP Servers. There is way to much unneeded stuff in the servers. We as a community would like to see how long new players stay and leave the servers. (Hours)

An other thing 

The server memory consumption is way too crazy. Including fights. There are way too many Issues with Fighting. Especially in Tanoa, don't blame the map for it, Its not a good excuse for the amount of mem-leaks and mass kicks. In general with Altis as well. 

Memory Leaks are your client (your PC) exceeding the maximum amount of RAM available for arma3.exe to use. Those have nothing to do with the Asylum servers. (if you doubt me, check the Arma 3 feedback tracker)

Mass kicks are from the exact same issue on the serverside.

Link to comment
1 hour ago, bamf said:

If you look over the last 30 days though, we average about 449 players per hour connected to the servers (24/7), and while there are times of day that the servers are not full (obviously) there are equal amounts of time that you have to spam to get in.  

Also, let's look at the going forward (or near past) instead of trying to rehash months ago.  We've made tremendous progress in the last week in particular with working on what we believe is the root cause of the issue - server memory consumption.  When the admin team is able to stay on top of the memory and proactively hard restart servers with near critical memory utilization, then the number of kicks is reduced to nearly zero.  

Servers are often crashing during Feds/banks is the amount of police vehicles being pulled due to cops dying and returning having an impact on server peformance? Also when suicide vests are detonated at rebels the server red chains for 30-45 seconds. Are these severely affecting the server?

Link to comment
Just now, Gnashes said:

Memory Leaks are your client (your PC) exceeding the maximum amount of RAM available for arma3.exe to use. Those have nothing to do with the Asylum servers. (if you doubt me, check the Arma 3 feedback tracker)

Mass kicks are from the exact same issue on the serverside.

Ways i'v prevented my Arma from mem-leaking https://gyazo.com/9d827e81ff7ed7f034474ccd726d8900

Link to comment
1 hour ago, bamf said:

We've made tremendous progress in the last week in particular with working on what we believe is the root cause of the issue - server memory consumption.  When the admin team is able to stay on top of the memory and proactively hard restart servers with near critical memory utilization, then the number of kicks is reduced to nearly zero.  

Ever considered maybe the reason behind a lot of memory issues is that Asylum is using a very old Altis Life framework and that it just can't handle anything more thrown at it? This was a theory Pentax came up with, it was also mentioned that the newer versions of Altis Life have multiple backend improvements that prevent/improve on memory issues like such hence the reason much 'newer' populated servers don't experience mass kicks. I don't know much about frameworks, but ever considered upgrading or actually developing your own framework? 

I know of 2 servers that are actually working on their own custom framework, it may be a time constraint and a lot of work, but maybe its something you guys can consider if 64bit doesn't show much luck.

Edited by Haych
Link to comment
33 minutes ago, Haych said:

Ever considered maybe the reason behind a lot of memory issues is that Asylum is using a very old Altis Life framework and that it just can't handle anything more thrown at it? This was a theory Pentax came up with, it was also mentioned that the newer versions of Altis Life have multiple backend improvements that prevent/improve on memory issues like such hence the reason much 'newer' populated servers don't experience mass kicks. I don't know much about frameworks, but ever considered upgrading or actually developing your own framework? 

I know of 2 servers that are actually working on their own custom framework, it may be a time constraint and a lot of work, but maybe its something you guys can consider if 64bit doesn't show much luck.

Our framework is really nothing like the original Altis Life framework at this point.  I don't think there would be any way to merge in a current framework with what we have since we (well Paratus) have basically re-written most of it anyway.  There was a time that we entertained scrapping basically all core components to re-write them as a new "framework", but that is a tremendous amount of effort to make something like that happen.  Arma 3 is really long in the tooth, so doing something like that at this point would likely not be a great investment in time.  

Again, I'm really hopeful that a 64-bit server can alleviate most of these issues.  The server just craps out when it gets close to 3GB of memory usage in the OS and 2GB of memory usage in game (that the server sees as "used").  Those are 32-bit limitations.  If the 64-bit server executable is not half baked and can actually ramp up, then the issue is more of server side FPS and making sure that we keep that at an acceptable level (to decrease things like de-sync).  

Link to comment
1 hour ago, Chewbacca said:

Yes ok there are more playing coming in but, they are always leaving because of stupid "Mem- Leaks" Nothing will change till that is fixed, I never got issues like this in other big time Altis/Tanoa RP Servers. There is way to much unneeded stuff in the servers. We as a community would like to see how long new players stay and leave the servers. (Hours)

An other thing 

The server memory consumption is way too crazy. Including fights. There are way too many Issues with Fighting. Especially in Tanoa, don't blame the map for it, Its not a good excuse for the amount of mem-leaks and mass kicks. In general with Altis as well. 

 

Mr. Linux likes this
Link to comment
1 minute ago, James.Anderson said:

Was playing on a German server before, very well developed, 230+ Players, In game VON was amazing, no kicks or lag :frog:... confused how Asylum can't manage such things :mellow:

Zero-One?

Funny. Seeing as we have their own graphs showing that they've suffered from mass kicks as well.

Link to comment
6 minutes ago, Gnashes said:
12 minutes ago, James.Anderson said:

Was playing on a German server before, very well developed, 230+ Players, In game VON was amazing, no kicks or lag :frog:... confused how Asylum can't manage such things :mellow:

Zero-One?

Funny. Seeing as we have their own graphs showing that they've suffered from mass kicks as well.

I don't remember the name, just that it was a very populated and developed german sever, played for a few hours and did not see any mass kicks, I just hope mass kicks will be fixed soon on asylum :) Asylum is my favourite server for altis life, playing hundreds of hours a month, guess you can say ime addicted:fail: deffiently not to crate keys tho , spent about $300, still no pilot coveralls :mellow:.....

 I have seen a reduction in mass kicks the past few weeks, especially for my self , but it's still a issue, hopefully a fix can be found soon, if anything can be done that is.

Edited by James.Anderson
Link to comment
Just now, James.Anderson said:

I don't remember the name, just that it was a very populated and developed german sever, played for a few hours and did not see any mass kicks, I just hope mass kicks will be fixed soon on asylum :) Asylum is my favourite server for altis life, playing hundreds of hours a month, guess you can say ime addicted:fail: deffiently not to crate keys tho , spent about $300, still no pilot coveralls :mellow:.....

 I have seen a reduction is mass kicks the past few weeks, especially for my self , but it's still a issue, hopefully a fix can be found soon, if anything can be done that is.

It's memory utilization. Crates were a huge step forward (even if we haven't actually FIXED the issue, just bandaided it).

The wanted system is the next thing.

Link to comment
4 minutes ago, Gnashes said:

It's memory utilization. Crates were a huge step forward (even if we haven't actually FIXED the issue, just bandaided it).

The wanted system is the next thing.

I understand completely, I have done a lot of coding and developing on Arma 3/2 before and have had similar issues, hopefully if They release 64 bit for servers it will help a lot, the servers should be able to use more RAM etc at that point right ? And that should help alot with current issues, and ofcourse finding ways to free up memory utilization server side from asylum will help greatly.

 

Or am I totally confused and drunk thinking about something else :D

Link to comment
Just now, James.Anderson said:

I understand completely, I have done a lot of coding and developing on Arma 3/2 before and have had similar issues, hopefully if They release 64 bit for servers it will help a lot, the servers should be able to use more RAM etc at that point right ? And that should help alot with current issues, and ofcourse finding ways to free up memory utilization server side from asylum will help greatly.

 

Or am I totally confused and drunk thinking about something else :D

Yep. Servers use 3GB of RAM, they start to mass kick from then until they hard restart every time they get full.

Link to comment
Guest
This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...