[infinispan-dev] Reading data differs from a WAR deployment or an EAR deployment

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[infinispan-dev] Reading data differs from a WAR deployment or an EAR deployment

Sanne Grinovero-3
Hi all,
please have a look into http://community.jboss.org/thread/171521?tstart=0

while the title and environment point to Hibernate Search, the trouble
is not related to Search but to Infinispan core; in short it seems
that it all works fine when using an application deployed in a WAR,
but when using EJBs instead the index is not updated; in fact it seems
the index is updated (correctly written to) but the clients don't see
the updates and are stuck as if a very long REPEATABLE_READ.

You might recall from past discussions that the Lucene Directory
 - requires batching
 - thou shall not configure a TransactionManager on the caches used by
the Lucene Directory
   or if you do, the transaction scope should be left to Lucene, not
having it join a containing transaction scope

So I didn't have time to try reproducing this issue yet, and likely
won't for a week at least, but is it possible that the batching layer
- which we know is calling into the DummyTransactionManager - is
actually connecting my batch to the real TransactionManager ?

Other possible explanations?

Regards,
Sanne
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Reading data differs from a WAR deployment or an EAR deployment

Manik Surtani

On 31 Aug 2011, at 11:33, Sanne Grinovero wrote:

So I didn't have time to try reproducing this issue yet, and likely
won't for a week at least, but is it possible that the batching layer
- which we know is calling into the DummyTransactionManager - is
actually connecting my batch to the real TransactionManager ?

Yes, it should fall back to the real Transaction Manager if one is present.  However it should suspend any existing transactions and start a new one, for the batch.



_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Reading data differs from a WAR deployment or an EAR deployment

Sanne Grinovero-3
2011/8/31 Manik Surtani <[hidden email]>:

>
> On 31 Aug 2011, at 11:33, Sanne Grinovero wrote:
>
> So I didn't have time to try reproducing this issue yet, and likely
> won't for a week at least, but is it possible that the batching layer
> - which we know is calling into the DummyTransactionManager - is
> actually connecting my batch to the real TransactionManager ?
>
> Yes, it should fall back to the real Transaction Manager if one is present.
>  However it should suspend any existing transactions and start a new one,
> for the batch.

Thanks for the clarification. So where else could I look for this
strange behaviour?
I guess I'll need an in-container test to debug, but I wonder if I'll
be able to reproduce this if I don't know the possible cause.

Sanne

_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Reading data differs from a WAR deployment or an EAR deployment

Galder Zamarreno

On Aug 31, 2011, at 3:44 PM, Sanne Grinovero wrote:

> 2011/8/31 Manik Surtani <[hidden email]>:
>>
>> On 31 Aug 2011, at 11:33, Sanne Grinovero wrote:
>>
>> So I didn't have time to try reproducing this issue yet, and likely
>> won't for a week at least, but is it possible that the batching layer
>> - which we know is calling into the DummyTransactionManager - is
>> actually connecting my batch to the real TransactionManager ?
>>
>> Yes, it should fall back to the real Transaction Manager if one is present.
>>  However it should suspend any existing transactions and start a new one,
>> for the batch.
>
> Thanks for the clarification. So where else could I look for this
> strange behaviour?
> I guess I'll need an in-container test to debug, but I wonder if I'll
> be able to reproduce this if I don't know the possible cause.

Might be tedious, but a TRACE log should help you figure out the sequence of events and try to mimmic those in a test?

>
> Sanne
>
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Reading data differs from a WAR deployment or an EAR deployment

Sanne Grinovero-3
update
it might just be a trivial error: duplicate jars on the classpath,
make the equals fail on the key when looking up a value from the
container.

This happened to myself once already and costed us some tear and
blood; I'm tempted to add a check for this to log a fat warning, but
this would have an overhead on a very hot path: to verify when the
class instances are different if their names where actually equal, to
be checked for each key comparison in happening during a cache lookup.

I'm thinking to add such check only if the debug/trace level are
enabled; still that would check for the debug level potentially many
times per query execution, do I'll check the logging level statically
.. better than nothing.

WDYT ?

Sanne
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Reading data differs from a WAR deployment or an EAR deployment

Sanne Grinovero-3
2011/9/1 Sanne Grinovero <[hidden email]>:

> update
> it might just be a trivial error: duplicate jars on the classpath,
> make the equals fail on the key when looking up a value from the
> container.
>
> This happened to myself once already and costed us some tear and
> blood; I'm tempted to add a check for this to log a fat warning, but
> this would have an overhead on a very hot path: to verify when the
> class instances are different if their names where actually equal, to
> be checked for each key comparison in happening during a cache lookup.
>
> I'm thinking to add such check only if the debug/trace level are
> enabled; still that would check for the debug level potentially many
> times per query execution, do I'll check the logging level statically
> .. better than nothing.
>
> WDYT ?
>
Extending the idea: today we've hit this issue on this specific key
implementation, but it's in no way specific to Lucene, it could happen
to anybody using Infinispan, with any key type both he's own custom
keys or our "framework keys" such as those used by Lucene or OGM.

Would it make sense to have an option which wraps all keys in the
container to do such a check? That would be able to quickly identify
classloading ambiguities, and being an Infinispan configuration option
it is a bit more intuitive to deal with than changing the behaviour
according to log levels.

Sanne
_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Reading data differs from a WAR deployment or an EAR deployment

Galder Zamarreno
In reply to this post by Sanne Grinovero-3

On Sep 1, 2011, at 2:03 PM, Sanne Grinovero wrote:

> update
> it might just be a trivial error: duplicate jars on the classpath,
> make the equals fail on the key when looking up a value from the
> container.

Classic classloading problem. Instance classes are only equals if both the classname and classloader are same.

> This happened to myself once already and costed us some tear and
> blood; I'm tempted to add a check for this to log a fat warning, but
> this would have an overhead on a very hot path: to verify when the
> class instances are different if their names where actually equal, to
> be checked for each key comparison in happening during a cache lookup.
>
> I'm thinking to add such check only if the debug/trace level are
> enabled; still that would check for the debug level potentially many
> times per query execution, do I'll check the logging level statically
> .. better than nothing.
>
> WDYT ?

I believe that trace/debug is enabled checks in JBoss Logging are pretty fast, so that could maybe work. I'd limit it to TRACE.

>
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Reply | Threaded
Open this post in threaded view
|

Re: [infinispan-dev] Reading data differs from a WAR deployment or an EAR deployment

Galder Zamarreno
In reply to this post by Sanne Grinovero-3

On Sep 1, 2011, at 2:10 PM, Sanne Grinovero wrote:

> 2011/9/1 Sanne Grinovero <[hidden email]>:
>> update
>> it might just be a trivial error: duplicate jars on the classpath,
>> make the equals fail on the key when looking up a value from the
>> container.
>>
>> This happened to myself once already and costed us some tear and
>> blood; I'm tempted to add a check for this to log a fat warning, but
>> this would have an overhead on a very hot path: to verify when the
>> class instances are different if their names where actually equal, to
>> be checked for each key comparison in happening during a cache lookup.
>>
>> I'm thinking to add such check only if the debug/trace level are
>> enabled; still that would check for the debug level potentially many
>> times per query execution, do I'll check the logging level statically
>> .. better than nothing.
>>
>> WDYT ?
>>
> Extending the idea: today we've hit this issue on this specific key
> implementation, but it's in no way specific to Lucene, it could happen
> to anybody using Infinispan, with any key type both he's own custom
> keys or our "framework keys" such as those used by Lucene or OGM.
>
> Would it make sense to have an option which wraps all keys in the
> container to do such a check? That would be able to quickly identify
> classloading ambiguities, and being an Infinispan configuration option
> it is a bit more intuitive to deal with than changing the behaviour
> according to log levels.

-1 to wrapping keys in container, that'd generate some extra garbage.

And I don't want to have to fiddle with configuration options to enable/disable this.

I'd rather switch TRACE logging on and get the warning that might help solve the issue.

>
> Sanne
> _______________________________________________
> infinispan-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache


_______________________________________________
infinispan-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/infinispan-dev