Missing objects due to restarts of the SOSS engine

[NadavB]

Hi,

In the last few days we experience many issues with missing objects in the cache. It happens every few hours.
We noticed in the SOSS logs that there were restarts. We saw these restarts in all the SOSS servers at the same time, so the whole cluster did reset.
I added a file with 3 events in the SOSS logs.
Can you please help us with it?

Thanks,
Nadav

 

[markw]

Hi Nadav,
This appears to be the same behavior that one of your associates (royial) reported last year:
https://community.scaleoutsoftware.com/threads/usage-of-soss-with-nuget.811/

We're interested to know what happened after the original report from last year--did the restarts continue occurring, or did they stop after some action on your side?

I see that we were looking into adding diagnostics to the service to into the service to investigate further, but royial went quiet while we were waiting to hear about some API usage around indexed properties:

• Do you store multiple types in the same cache? If so, do you use inheritance such that properties on the base class are indexed?
• Do any of your classes use more than eight indexed properties marked with HashIndexPriority.HighPriorityHashable?
• Is it possible that you have different versions of the same class with different indexed properties in a cache? (For example, perhaps you have multiple versions of your app running side-by-side?)

 

[NadavB]

Hi Mark,

After the original report the restarts continued but their frequency reduced without any action from our side. Also this time, the same behavior - a few days with many restarts and somehow it got stable.

We do store multiple types in the same cache. We use inheritance, but we don't use SossIndex at all.
Each class that we store in the cache can have just 1 version, when we change the version we remove all the old objects and load the new ones.

Even though it got stable for now, we would like to know what we can do next time it will happen. Thanks!

 

[markw]

We'll change the service so that it will just log the problem instead of restarting if/when the condition occurs. The next release will be out within the next two weeks.