nsHebrewProber Class Reference

#include <nsHebrewProber.h>

Inheritance diagram for nsHebrewProber:

[legend]

Collaboration diagram for nsHebrewProber:

[legend]

Public Member Functions
	nsHebrewProber (void)
virtual	~nsHebrewProber (void)
virtual nsProbingState	HandleData (const char *aBuf, PRUint32 aLen)
virtual const char *	GetCharSetName ()
virtual void	Reset (void)
virtual nsProbingState	GetState (void)
virtual float	GetConfidence (void)
virtual void	SetOpion ()
void	SetModelProbers (nsCharSetProber *logicalPrb, nsCharSetProber *visualPrb)
Static Protected Member Functions
static PRBool	isFinal (char c)
static PRBool	isNonFinal (char c)
Protected Attributes
PRInt32	mFinalCharLogicalScore
PRInt32	mFinalCharVisualScore
char	mPrev
char	mBeforePrev
nsCharSetProber *	mLogicalProb
nsCharSetProber *	mVisualProb

---

Detailed Description

Definition at line 45 of file nsHebrewProber.h.

---

Constructor & Destructor Documentation

nsHebrewProber::nsHebrewProber

(

void

)

[inline]

Definition at line 44 of file nsHebrewProber.h.

Referenced by ~nsHebrewProber().

: public nsCharSetProber

Here is the caller graph for this function:

virtual nsHebrewProber::~nsHebrewProber

(

void

)

[inline, virtual]

Definition at line 46 of file nsHebrewProber.h.

References GetCharSetName(), HandleData(), mLogicalProb, mVisualProb, nsHebrewProber(), and Reset().

{

Here is the call graph for this function:

---

Member Function Documentation

const char * nsHebrewProber::GetCharSetName

(

)

[virtual]

Implements nsCharSetProber.

Definition at line 145 of file nsHebrewProber.cpp.

References nsCharSetProber::GetConfidence(), LOGICAL_HEBREW_NAME, mFinalCharLogicalScore, mFinalCharVisualScore, MIN_FINAL_CHAR_DISTANCE, MIN_MODEL_DISTANCE, mLogicalProb, mVisualProb, and VISUAL_HEBREW_NAME.

Referenced by ~nsHebrewProber().

{
  // If the final letter score distance is dominant enough, rely on it.
  PRInt32 finalsub = mFinalCharLogicalScore - mFinalCharVisualScore;
  if (finalsub >= MIN_FINAL_CHAR_DISTANCE) 
    return LOGICAL_HEBREW_NAME;
  if (finalsub <= -(MIN_FINAL_CHAR_DISTANCE))
    return VISUAL_HEBREW_NAME;

  // It's not dominant enough, try to rely on the model scores instead.
  float modelsub = mLogicalProb->GetConfidence() - mVisualProb->GetConfidence();
  if (modelsub > MIN_MODEL_DISTANCE)
    return LOGICAL_HEBREW_NAME;
  if (modelsub < -(MIN_MODEL_DISTANCE))
    return VISUAL_HEBREW_NAME;

  // Still no good, back to final letter distance, maybe it'll save the day.
  if (finalsub < 0) 
    return VISUAL_HEBREW_NAME;

  // (finalsub > 0 - Logical) or (don't know what to do) default to Logical.
  return LOGICAL_HEBREW_NAME;
}

Here is the call graph for this function:

Here is the caller graph for this function:

virtual float nsHebrewProber::GetConfidence

(

void

)

[inline, virtual]

Implements nsCharSetProber.

Definition at line 53 of file nsHebrewProber.h.

{ return (float)0.0; }

nsProbingState nsHebrewProber::GetState

(

void

)

[virtual]

Implements nsCharSetProber.

Definition at line 181 of file nsHebrewProber.cpp.

References eDetecting, eNotMe, nsCharSetProber::GetState(), mLogicalProb, and mVisualProb.

Referenced by HandleData().

{
  // Remain active as long as any of the model probers are active.
  if ((mLogicalProb->GetState() == eNotMe) && (mVisualProb->GetState() == eNotMe))
    return eNotMe;
  return eDetecting;
}

Here is the call graph for this function:

Here is the caller graph for this function:

nsProbingState nsHebrewProber::HandleData	(	const char *	aBuf,
		PRUint32	aLen
	)			[virtual]

HandleData Final letter analysis for logical-visual decision. Look for evidence that the received buffer is either logical Hebrew or visual Hebrew. The following cases are checked: 1) A word longer than 1 letter, ending with a final letter. This is an indication that the text is laid out "naturally" since the final letter really appears at the end. +1 for logical score. 2) A word longer than 1 letter, ending with a Non-Final letter. In normal Hebrew, words ending with Kaf, Mem, Nun, Pe or Tsadi, should not end with the Non-Final form of that letter. Exceptions to this rule are mentioned above in isNonFinal(). This is an indication that the text is laid out backwards. +1 for visual score 3) A word longer than 1 letter, starting with a final letter. Final letters should not appear at the beginning of a word. This is an indication that the text is laid out backwards. +1 for visual score.

The visual score and logical score are accumulated throughout the text and are finally checked against each other in GetCharSetName(). No checking for final letters in the middle of words is done since that case is not an indication for either Logical or Visual text.

The input buffer should not contain any white spaces that are not (' ') or any low-ascii punctuation marks.

Implements nsCharSetProber.

Definition at line 109 of file nsHebrewProber.cpp.

References eDetecting, eNotMe, GetState(), isFinal(), isNonFinal(), mBeforePrev, mFinalCharLogicalScore, mFinalCharVisualScore, and mPrev.

Referenced by ~nsHebrewProber().

{
  // Both model probers say it's not them. No reason to continue.
  if (GetState() == eNotMe)
    return eNotMe;

  const char *curPtr, *endPtr = aBuf+aLen;
  char cur;

  for (curPtr = (char*)aBuf; curPtr < endPtr; ++curPtr)
  {
    cur = *curPtr;
    if (cur == ' ') // We stand on a space - a word just ended
    {
      if (mBeforePrev != ' ') // *(curPtr-2) was not a space so prev is not a 1 letter word
      {
        if (isFinal(mPrev)) // case (1) [-2:not space][-1:final letter][cur:space]
          ++mFinalCharLogicalScore;
        else if (isNonFinal(mPrev)) // case (2) [-2:not space][-1:Non-Final letter][cur:space]
          ++mFinalCharVisualScore;
      }
    }
    else  // Not standing on a space
    {
      if ((mBeforePrev == ' ') && (isFinal(mPrev)) && (cur != ' ')) // case (3) [-2:space][-1:final letter][cur:not space]
        ++mFinalCharVisualScore;
    }
    mBeforePrev = mPrev;
    mPrev = cur;
  }

  // Forever detecting, till the end or until both model probers return eNotMe (handled above).
  return eDetecting;
}

Here is the call graph for this function:

Here is the caller graph for this function:

PRBool nsHebrewProber::isFinal

(

char

c

)

[static, protected]

Definition at line 64 of file nsHebrewProber.cpp.

References FINAL_KAF, FINAL_MEM, FINAL_NUN, FINAL_PE, and FINAL_TSADI.

Referenced by HandleData().

{
  return ((c == FINAL_KAF) || (c == FINAL_MEM) || (c == FINAL_NUN) || (c == FINAL_PE) || (c == FINAL_TSADI));
}

Here is the caller graph for this function:

PRBool nsHebrewProber::isNonFinal

(

char

c

)

[static, protected]

Definition at line 69 of file nsHebrewProber.cpp.

References NORMAL_KAF, NORMAL_MEM, NORMAL_NUN, and NORMAL_PE.

Referenced by HandleData().

{
  return ((c == NORMAL_KAF) || (c == NORMAL_MEM) || (c == NORMAL_NUN) || (c == NORMAL_PE));
  // The normal Tsadi is not a good Non-Final letter due to words like 
  // 'lechotet' (to chat) containing an apostrophe after the tsadi. This 
  // apostrophe is converted to a space in FilterWithoutEnglishLetters causing 
  // the Non-Final tsadi to appear at an end of a word even though this is not 
  // the case in the original text.
  // The letters Pe and Kaf rarely display a related behavior of not being a 
  // good Non-Final letter. Words like 'Pop', 'Winamp' and 'Mubarak' for 
  // example legally end with a Non-Final Pe or Kaf. However, the benefit of 
  // these letters as Non-Final letters outweighs the damage since these words 
  // are quite rare.
}

Here is the caller graph for this function:

void nsHebrewProber::Reset

(

void

)

[virtual]

Implements nsCharSetProber.

Definition at line 170 of file nsHebrewProber.cpp.

References mBeforePrev, mFinalCharLogicalScore, mFinalCharVisualScore, and mPrev.

Referenced by ~nsHebrewProber().

{
  mFinalCharLogicalScore = 0;
  mFinalCharVisualScore = 0;

  // mPrev and mBeforePrev are initialized to space in order to simulate a word 
  // delimiter at the beginning of the data
  mPrev = ' ';
  mBeforePrev = ' ';
}

Here is the caller graph for this function:

void nsHebrewProber::SetModelProbers	(	nsCharSetProber *	logicalPrb,
		nsCharSetProber *	visualPrb
	)			[inline]

Definition at line 56 of file nsHebrewProber.h.

Referenced by nsSBCSGroupProber::nsSBCSGroupProber().

                                        { return (float)0.0; }

Here is the caller graph for this function:

virtual void nsHebrewProber::SetOpion

(

)

[inline, virtual]

Implements nsCharSetProber.

Definition at line 54 of file nsHebrewProber.h.

{ return (float)0.0; }

---

Field Documentation

char nsHebrewProber::mBeforePrev [protected]

Definition at line 70 of file nsHebrewProber.h.

Referenced by HandleData(), and Reset().

PRInt32 nsHebrewProber::mFinalCharLogicalScore [protected]

Definition at line 67 of file nsHebrewProber.h.

Referenced by GetCharSetName(), HandleData(), and Reset().

PRInt32 nsHebrewProber::mFinalCharVisualScore [protected]

Definition at line 67 of file nsHebrewProber.h.

Referenced by GetCharSetName(), HandleData(), and Reset().

nsCharSetProber* nsHebrewProber::mLogicalProb [protected]

Definition at line 73 of file nsHebrewProber.h.

Referenced by GetCharSetName(), GetState(), and ~nsHebrewProber().

char nsHebrewProber::mPrev [protected]

Definition at line 70 of file nsHebrewProber.h.

Referenced by HandleData(), and Reset().

nsCharSetProber * nsHebrewProber::mVisualProb [protected]

Definition at line 73 of file nsHebrewProber.h.

Referenced by GetCharSetName(), GetState(), and ~nsHebrewProber().

---

The documentation for this class was generated from the following files:

• nsHebrewProber.h
• nsHebrewProber.cpp