Error - Cannot access a disposed object

[YardenK]

Hello,
We're using the latest SOSS server (5.10.9) and use the NuGet in our code (nuget version - 3.1.4).

We are seeing this error in the event viewer of our server:

Message: Cannot access a disposed object.
Object name: 'The ThreadLocal object has been disposed.'.
StackTrace: at System.Threading.ThreadLocal`1.GetValueSlow()
at Scaleout.Client.KeyEncoding.LegacyStringKeyEncoder.Encode(String key)
at Scaleout.Client.Cache`2.Read(TKey key, ReadOptions options, CancellationToken cancellationToken)

Is it a known issue? Is there something we are doing wrong that causes this error?

Thanks!

 

[markw]

Hi YardenK. This is not a known issue.

• Is the exception thrown when your app first starts up or does is only occur after your app has been running for a long time?
• What kind of threading model does your app use? Is this just a regular ASP.NET app serving web requests, or does your app start a lot of its own background worker threads and then let them exit?
• Is this exception sporadic, or does it happen with every call?
• What is the exact .NET runtime that you use?

 

[YardenK]

Hi Mark, thanks for replying.

• Is the exception thrown when your app first starts up or does is only occur after your app has been running for a long time? It occurs after the app is up and running.
• What kind of threading model does your app use? Is this just a regular ASP.NET app serving web requests, or does your app start a lot of its own background worker threads and then let them exit? We do have background threads, also windows services that run and use the cache.
• Is this exception sporadic, or does it happen with every call? it is not on every call but it is not very rare. We have 10 machines and I see this exception at least once a day on every machine.
• What is the exact .NET runtime that you use? We are using .net framework 4.7.2.

Any idea what can cause this? Is this something we are doing wrong? What else can I check?

Thanks!

 

[YardenK]

BTW - seeing a different stack trace now in the event viewer, if it helps:
Exception: System.ObjectDisposedException

Message: Cannot access a disposed object.
Object name: 'System.Net.Sockets.NetworkStream'.

StackTrace: at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.BufferedStream.Flush()
at Scaleout.Client.Net.Messaging.ReadRequest.WriteToStream(Stream stream, ObjectKey key, VersionToken version, ReadFlags flags, LockToken lockToken, Boolean isLocalClient)
at Scaleout.Client.Net.Store.<>c__DisplayClass24_0.<Read>b__0(Stream stream)
at Scaleout.Client.Net.Store.DoRequest(ObjectKey key, IPAddress preferredHost, Func`2 requestFunc, CancellationToken ct)
at Scaleout.Client.Net.Store.Read(ObjectKey key, LockToken lockToken, VersionToken version, ReadFlags flags, Byte[] keyList, Nullable`1 mmfKey, CancellationToken cancellationToken)
at Scaleout.Client.Cache`2.ReadInternal(ObjectKey objectKey, ReadOptions options, ReadFlags flags, LockToken lockToken, AccessOperation traceOp, TKey key, CancellationToken ct)
at Scaleout.Client.Cache`2.Read(TKey key, ReadOptions options, CancellationToken cancellationToken)

 

[markw]

Any idea what can cause this? Is this something we are doing wrong? What else can I check?

Thank you for those answers. I do not yet see a usage pattern or a mistake that you could make that would explain this -- the ObjectDisposedException is complaining about long-lived thread-local resource that the ScaleOut library never disposes. We need to learn more about your environment.

• Could this be occurring while your application is shutting down (for example, during an IIS app pool recycle)? Please check to see whether the timing of this exception aligns with your app pool's recycling schedule (by default, IIS will recycle your app every 29 hours).

BTW - seeing a different stack trace now in the event viewer, if it helps:
Exception: System.ObjectDisposedException
Message: Cannot access a disposed object.
Object name: 'System.Net.Sockets.NetworkStream'.

This, too, is very suspicious. It's coming out of a completely unrelated resource in a different layer of the library. The library's network connections are only disposed after the membership of your ScaleOut cluster changes, and never right in the middle of a read operation.

• Are these exceptions being thrown out of an ASP.NET web app, the Windows Service that you mentioned, or both?
• If it's coming out of a web app, how many different web apps do you have assigned to the same app pool that's hosting the app?
• Are you aware of any management/maintenance activity occurring on the ScaleOut cluster during the time of this exception?