Shadow Item Count Optimization:

For ideas on how to make Second Age a better shard. Can it get any better? Maybe.
Forum rules
Posts in this forum are expected to be constructive, realistic and civil. Inflamatory or off topic posts will be removed.
User avatar
Hemperor
UOSA Subscriber!
UOSA Subscriber!
Posts: 4368
Joined: Sat Jul 19, 2008 9:15 am

Re: Shadow Item Count Optimization:

Post by Hemperor »

Faust wrote:
Faust wrote:This would only be possible with items containing the exact same property listings of course.
You do make a good point on the creation date property though.

The data could be placed into an array or list, but like you mentioned that would start getting more complex than what it's worth.

I could just imagine one mishap or error leading to dupe bugs or the loss of the shadow items altogether.
Sorry, I didn't spot that. Anyways, I know there are a number of properties for every single base item in RunUO, but I also know Derrick has some his own optimisation so what properties each general item has really only he knows.

Oh yeah, just came to me, things like durability etc which are across all tools... It's just not feasible.
Image

[22:26] <ian> why am i making 3750 empty kegs
[22:27] <ian> 1125000 for 3750 empty kegs
----------------------------------------
[10:44] <ian> a good cat is a dead cat

Hicha
UOSA Donor!!
UOSA Donor!!
Posts: 2264
Joined: Tue May 05, 2009 10:03 am
Location: out selling permits

Re: Shadow Item Count Optimization:

Post by Hicha »

If the issue is due to inactive players, would it be possible to archive player accounts and completely remove the player and all of its banked/backpack items from the active save database? Then if a player returns from a long haitus, they can just have Derrick re-initiate their information into the core database?
Image
"I consider most of you NPC's that inhabit the single player game that I am here to enjoy." - MatronDeWinter

User avatar
Hemperor
UOSA Subscriber!
UOSA Subscriber!
Posts: 4368
Joined: Sat Jul 19, 2008 9:15 am

Re: Shadow Item Count Optimization:

Post by Hemperor »

Hicha wrote:If the issue is due to inactive players, would it be possible to archive player accounts and completely remove the player and all of its banked/backpack items from the active save database? Then if a player returns from a long haitus, they can just have Derrick re-initiate their information into the core database?
I believe all mobiles and items are loaded into memory from server startup. With your idea, Derrick would have to somehow add the person back into the "live" database and then restart the server. I don't think its really feasible either for Derrick to modify everything to be able to "load" at runtime either.

Your idea would certainly help but I think for it to not impede on everyone else that people would have to inform Derrick and on next server restart they would be back, but then this also potentially adds a lot more maintenance and work for Derrick than I'm sure he's interested in.

Hate to be Mr. Bad News, these are all just my opinions, but Hicha and Faust's suggestions so far are on the right track in that to have any possibility of solving these issues we really have to think outside the box.
Image

[22:26] <ian> why am i making 3750 empty kegs
[22:27] <ian> 1125000 for 3750 empty kegs
----------------------------------------
[10:44] <ian> a good cat is a dead cat

User avatar
Blaise
UOSA Donor!!
UOSA Donor!!
Posts: 2466
Joined: Mon Feb 14, 2011 12:13 am
Location: Trammel

Re: Shadow Item Count Optimization:

Post by Blaise »

If possible, archiving the entire account would be great.
A custom message on login indicating "Your account has been archived due to inactivity. Please email derrick@uosecondage.com to request restoration"
At which point, queue up the account for restoration on the morning reboot.

Granted, I have no idea how complicated that would be to implement, but the theory sounds a lot more feasible.


"or the loss of the shadow items altogether."


OH NOEZ!! :P
Est Sularus oth Mithas

User avatar
Faust
Posts: 6247
Joined: Mon Sep 22, 2008 7:01 pm

Re: Shadow Item Count Optimization:

Post by Faust »

Hicha, that was my initial suggestion in the thread that started the whole discussion.
Faust wrote: How viable would this solution be for optimization?

We know RunUO saves/loads data in a text document data style structure. The system could be restructured a little based on specific character inactivity instead of the entire account itself since this would include those characters that are rarely/never played on active accounts too. This new system would not DELETE anything and would actually store inactive characters/items/etc... on a secondary file system using the exact same methods RunUO uses with just some tweaked methodologies. The secondary file system would store inactive characters/items and would serve as a type of retrieval system that the server does not look at unless someone tries to log onto the inactive character. When someone attempts to log into one of the characters stored in the secondary system it would simply extract all the information during the login process and pull it into the actual saved data files.

This should solve the majority of the problems that are due to inactivity while sticking to the original shard idea that your account information will always remain intact no matter the inactivity associated with it. The only concern on my part is the server load when someone attempts to log into a character with a lot of data during regular shard hours but it may not even be noticable in reality though.
Derrick's follow up response can be read there.

Hicha
UOSA Donor!!
UOSA Donor!!
Posts: 2264
Joined: Tue May 05, 2009 10:03 am
Location: out selling permits

Re: Shadow Item Count Optimization:

Post by Hicha »

Faust wrote:Hicha, that was my initial suggestion in the thread that started the whole discussion.
Faust wrote: How viable would this solution be for optimization?

We know RunUO saves/loads data in a text document data style structure. The system could be restructured a little based on specific character inactivity instead of the entire account itself since this would include those characters that are rarely/never played on active accounts too. This new system would not DELETE anything and would actually store inactive characters/items/etc... on a secondary file system using the exact same methods RunUO uses with just some tweaked methodologies. The secondary file system would store inactive characters/items and would serve as a type of retrieval system that the server does not look at unless someone tries to log onto the inactive character. When someone attempts to log into one of the characters stored in the secondary system it would simply extract all the information during the login process and pull it into the actual saved data files.

This should solve the majority of the problems that are due to inactivity while sticking to the original shard idea that your account information will always remain intact no matter the inactivity associated with it. The only concern on my part is the server load when someone attempts to log into a character with a lot of data during regular shard hours but it may not even be noticable in reality though.
Derrick's follow up response can be read there.
Weird to think I came up with the same recommendation as you. =X This a Step Bothers moment? lol
Image
"I consider most of you NPC's that inhabit the single player game that I am here to enjoy." - MatronDeWinter

LKP
Posts: 48
Joined: Thu May 17, 2012 9:49 am
Location: CT, USA
Contact:

Re: Shadow Item Count Optimization:

Post by LKP »

I was just thinking of something similar. I know the staff already has something in the works that blows alll of our uninformed suggestions away, but it's still fun to speculate. So the thought is this: if it ain't changed, don't save it. There's no need to update the data for a given player mobile if it hasn't been logged in since the last save. This of course extends to all items in its backpack and bank. Maybe something similar can be done for houses if no playermobile has been in that house during that save period. But maybe it already works this way, maybe this won't be very effective, or maybe it's not practical. Just shooting in the dark here.

User avatar
Faust
Posts: 6247
Joined: Mon Sep 22, 2008 7:01 pm

Re: Shadow Item Count Optimization:

Post by Faust »

That is a very interesting idea LKP.

I have no idea if the serialization method functions in this manner or not for saving data. This would still come down to the item properties and if there are any of the properties that are constantly changing(can't think of any of hand besides mobiles). Would have to see if Derrick has something like this in place already or not though. Hemperor is right about thinking outside of the box though. These are the type of ideas that need to be tossed around and discussed.

User avatar
Blaise
UOSA Donor!!
UOSA Donor!!
Posts: 2466
Joined: Mon Feb 14, 2011 12:13 am
Location: Trammel

Re: Shadow Item Count Optimization:

Post by Blaise »

I have no idea what the data looks like that's being backed up, or if this could apply at all, but I once setup mirrored SQL servers in two separate geographic locations. I utilized an HP software product called Storage Mirroring Server. This software replicated the data between the two storage arrays constantly and with low bandwidth consumption (irrelevant if in the same datacenter really).
If something like that could function with the data set for UOSA, then no saving ever needs to be run on the production system. Data is replicated live to the alternate storage (at a bit-by-bit level) and that information can be saved/backed up without impacting performance of the source data set.
Est Sularus oth Mithas

User avatar
Faust
Posts: 6247
Joined: Mon Sep 22, 2008 7:01 pm

Re: Shadow Item Count Optimization:

Post by Faust »

Based on my understanding RunUO uses a "SQL based" file system that stores the properties of each mobile/item that are saved during the serialization method that is called at save time. The data is "deserialized" when that method is called during server start. When the server pauses for a save all the data is being serialized.

Please correct me if I am wrong... :)

User avatar
Blaise
UOSA Donor!!
UOSA Donor!!
Posts: 2466
Joined: Mon Feb 14, 2011 12:13 am
Location: Trammel

Re: Shadow Item Count Optimization:

Post by Blaise »

The system I setup had multiple paths of recovery, should a failure occur.
I was taking nightly backups of the primary, to a data store on the local network, which replicated to it's sibling in the data center. Then the drive arrays were also replicated live between local and the data center. In effect, I could restore to the previous day (up to 60 days back) any time, or if urgent, I could restore up to the minute of failure by cutting over to the live replica.
Est Sularus oth Mithas

Vilenarios
UOSA Donor!!
UOSA Donor!!
Posts: 226
Joined: Mon Jun 21, 2010 7:27 pm

Re: Shadow Item Count Optimization:

Post by Vilenarios »

Blaise wrote:The system I setup had multiple paths of recovery, should a failure occur.
I was taking nightly backups of the primary, to a data store on the local network, which replicated to it's sibling in the data center. Then the drive arrays were also replicated live between local and the data center. In effect, I could restore to the previous day (up to 60 days back) any time, or if urgent, I could restore up to the minute of failure by cutting over to the live replica.
Sounds like some pro UO DR.

If.it were only that easy in an enterprise

User avatar
Blaise
UOSA Donor!!
UOSA Donor!!
Posts: 2466
Joined: Mon Feb 14, 2011 12:13 am
Location: Trammel

Re: Shadow Item Count Optimization:

Post by Blaise »

Vilenarios wrote:
Blaise wrote:The system I setup had multiple paths of recovery, should a failure occur.
I was taking nightly backups of the primary, to a data store on the local network, which replicated to it's sibling in the data center. Then the drive arrays were also replicated live between local and the data center. In effect, I could restore to the previous day (up to 60 days back) any time, or if urgent, I could restore up to the minute of failure by cutting over to the live replica.
Sounds like some pro UO DR.

If.it were only that easy in an enterprise
Haha, this was for an instance of MS Dynamics on an SQL 2008/Win2008 server. :)
I was almost sad I never got to test it, aside from restoring backups in a sandbox for validation.
Est Sularus oth Mithas

Post Reply