Bitwise Logical Operations in CA-Visual Objects

Patrick Lazzarotto

Introduction
Systems programming applications require low-level control over and access into the computer memory. s Bit-level programming, "bit twiddling" in the vernacular, is a set of techniques for managing and manipulating individual bits within a memory location. Modern operating systems provide direct access to operating system services through APIs that make extensive use of bits for storing binary settings. A solid understanding of bitwise operations is crucial for directly accessing operating services, such as those available via the Microsoft Windows APIs. Operating systems, however, are not the only place where bitlevel programming techniques are useful. As shall be demonstrated, a solid understanding of bit-level programming is a useful addition to every programmer's arsenal. This article considers a subset of bit-level programming techniques collectively known as bit-mask operations and provides implementations and examples in the CA-Visual Objects 2 language. After a review of the basics, the logical bitwise operators, OR, AND, XOR and NOT are introduced. Then the bitlevel programming technique known as bit masking is explained and demonstrated. A bit mask operations API is implemented in CA-Visual Objects to provide a simple interface for these services and shows how easy it can be to program at the bit-level. The article concludes with an exploration of the application design implications of bit-mask operations via a real-world case study.

Basic Bits
Most contemporary computers store data as binary integers and use twos complement notation where the high order bit acts as a sign bit for the representation of negative numbers. The binary number system has a base of two, providing two binary digits, zero and one. Each digit of a binary integer is called a bit, shorthand for binary digit. On Intel-based computers, a byte is composed of eight bits. A byte stored at an address in computer memory can be conceptualized as a string, or bit map of eight binary digits. 00110100 The rightmost bit is called the least significant or low-order bit, while the leftmost bit is the most significant or high-order bit. If the string of bits represents an integer, the rightmost bit represents 2^^0, the next least significant bit represents 2^^1, then 2^^3, and so on. Therefore, the binary number 00110100 represents the value 2^^3 + 2^^5 + 2^^6, or 100 (4 + 32 + 64) in decimal notation. A single byte can store is 2^^8, or 256, unique values. An unsigned integer with n bits can have values ranging from zero through 2^^n - 1. When integer values are signed using twos complement notation, the largest positive number that can be stored into n bits is 2^^(n-1) 1 (127), and the largest negative number that can be stored in n bits is 2^^(n-1) (128).

Bitwise Operators
Use of bitwise operators is common programming practice at the operating system level. The bitwise operators provide access to individual bits and groups of bits in memory. Bitwise operators manipulate the bits within integer data types. In C, character handling sometimes requires operating on the bits within the character byte. However, this application is less useful for higher-level languages, like C++ and CA-Visual Objects, which have a wider variety of native data types and provide for user-definable classes. Figure 1 shows the ANSI Standard C/C++ bitwise operators and the equivalent in CA-Visual Objects.
Binary Logical Operation Bitwise right shift Bitwise left shift Bitwise inclusive OR Bitwise exclusive OR Bitwise AND Ones' complement C/C++ >> << | ^ & ~ CA-Visual Objects >> << _OR() _XOR() _AND() _NOT()

Figure 1: Bitwise logical operators in ANSI C/C++ and CA-Visual Objects 2.

There are two types of bitwise operators: the bitwise shift operators and the bitwise logical operators. The Bitwise Shift Operators The bitwise left and right shift operators provide swift and efficient multiplication and division by powers of two, respectively, when used with unsigned values. Aside from their obvious mathematical uses, the bitwise shift operators are sometimes useful for incrementing pointers when walking nested structures. The Bitwise Logical Operators The four bitwise logical operators, inclusive OR, AND, exclusive OR (XOR) and ones' complement (NOT), are a focus of this article. The binary bitwise logical operators, OR, AND and XOR perform bit-bybit comparisons of the two operands. The return value is a set of bits with each bit defined according to the appropriate truth table (see Figure 2). Bitwise OR, AND, and XOR operations are commutative so the order of the operands is not important. Ones' complement, or bitwise negation, is a unary operator, performing a bitwise inversion on each bit in the operand. Bitwise negation is supported by the ~ operator in C/C++. CAVisual Objects' equivalent, the _NOT() operator, exists, but is currently an undocumented feature.
bit 1 0 0 1 1 bit 2 0 1 0 1 OR 0 1 1 1 AND 0 0 0 1 XOR 0 1 1 0

Figure 2: Bitwise OR, AND and XOR truth tables for a single bit.

Bitwise Inclusive OR The bitwise inclusive OR operator sets a bit to one if either of the corresponding bits in its operands is one, and to zero if both corresponding bits are zeros. 0b00010000 0b00000001 OR 0b00010001 Bitwise AND The bitwise AND operator produces a new value by making a bit-by-bit comparison between two operands. For each bit position, the bitwise AND operator sets a bit to one if the corresponding bits in its operands are one, and to zero if the bits differ or both are zeros.

0b010010011 0b001111011 AND 0b000010011 Bitwise Exclusive OR (XOR) The bitwise exclusive OR operator (XOR) compares each corresponding bit in the two operands and, for each bit position, sets the bit to one where the corresponding bits in its operands are different, and to zero if they are the same. 0b01000001 0b00010001 XOR 0b01010000 Bitwise Negation (NOT) The bitwise negation, or ones' complement, operator is a unary operator it takes only one parameter. It inverts each bit in the operand, changing all the ones to zeros and zeros to ones.
b1 0 0 1 1 NOT 1 1 0 0

Figure 3: Truth table for bitwise logical XOR.

NOT( 0b00000100 ) == 0b11111011

// TRUE

The Bitwise Logical Operators in CA-Visual Objects 2 CA-Visual Objects provides a set of operators specifically designed for performing operations on individual bits roughly equivalent to those available in C/C++. Like C/C++, CA-Visual Objects' bitwise operators act on integer numeric data types. In CA-Visual Objects, the operands and return values of the _OR(), _AND() and _XOR() operators may be values of one of the following data types: INT, SHORTINT, LONGINT, WORD or DWORD. Note that unlike C/C++, CA-Visual Objects does not support the BYTE data type as an operand for the bitwise logical operators. New to CA-Visual Objects 2.0, the binary bitwise operators, _AND(), _OR() and _XOR(), take two or more operands which avoids having to nest successive bitwise operations. In CA-Visual Objects 1.x the following statement would be used to OR 3 operands, _OR( dword1, _OR( dword2, dword3 ) ) The CA-Visual Objects 2.x syntax is more convenient: _OR( dword1, dword2, dword3 ) CA-Visual Objects implements its OR, AND, XOR and undocumented NOT bitwise operators as runtime functions. The runtime implementation of bitwise operators in the CA-Visual Objects language means that, unlike C/C++, execute and assign compound bitwise operations cannot be defined to the compiler by overloading the equals operator.
Compound Bitwise Operator Bitwise OR and assign Bitwise AND and assign Bitwise XOR and assign Ones' complement and assign Bitwise right shift and assign Bitwise left shirt and assign C/C++ =| =& =^ ==>> =<< CA-Visual Objects += not supported not supported not supported =>> =<<

Figure 4: Comparison of compound, execute and assign bitwise operations in C/C++ and CA-Visual Objects.

In CA-Visual Objects 2.0, the plus sign (+) can be used to express a bitwise logical OR between two values because the operation simply is addition. Flag_1 := 0b01 Flag_2 := 0b10 Options := FLAG_1 + FLAG_2 ? Options // 1 decimal // 2 decimal // 1 + 2 // 0b11 or 3 decimal

Use of this 'overloaded' plus operator is common practice for the setting of window styles. oTextBox:Type := MB_ICONHAND + MB_YESNO This use of the plus operator is required for CA-Visual Objects constants (DEFINE statements) where the binary bitwise operators a re not supported due to their runtime implementation.

Bit Flags and Bit Masking Operations

Bit masks are founded on the idea that an area of computer memory may be viewed as a set of Boolean flags, a bank of binary switches having two possible states, on or off. A set of bits provides a convenient and efficient memory structure to represent a set of Boolean values. The set of bits in memory that constitute an integer data type may be manipulated and passed as an atomic data item, and the bits within the data item may be accessed individually, bit by bit, with the bitwise logical operators. The logical bitwise operators are often used in conjunction with the concepts of bit flags and bit masks. A byte used as a field of Boolean values is referred to as a flag byte and the individual bits as flag bits. A bit mask is simply a pattern of bits. Bit masks map a pattern of bits to its Boolean counterpart in the abstraction being represented. By convention, a mask for a particular bit location is that bit set to one, and all other bits set to zero. Figure 5 clarifies why the technique is called bit masking. For a bitwise AND, the zeros in MASK, hide the corresponding bits in the memory variable, flag, such that the result contains only the bits that are ones in both operands. You can think of the zeros in the mask as being opaque and the ones as being transparent. Thus the C language expression, flags & MASK, or _AND( flags, MASK ) in CA-Visual Objects syntax, is like covering the bit pattern of flags with that of MASK so that only the bits under the ones of MASK are visible.
MASK flag result 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 1 1 1 0 0 0 AND

Figure 5: Bit mask

A set of constants provides a convenient and intuitive way to refer to each bit position. There are also times when successive bitwise logical operations can be combined into a single operation by defining masks that are combinations of the primitive set of masks. Ob00000000 EMPTY_MASK 0b00000001 MASK_1 0b00000010 MAKS_2 0b00000100 MAKS_3 ... ... 0b11111111 ALL_MASK Bit mask constants may be defined in binary, decimal or hexadecimal notation; whichever is most convenient for the task at hand. Hexadecimal notation is a more compact, and sometimes more convenient, way to write binary values since each hexadecimal digit refers to four bits. Figure 6 shows a set of bit masks with values expressed in binary, decimal, and hexadecimal notation.

Bit Position and/or Bit Mask

Binary Notation (Bit Mask)

Decimal Notation (Decimal Mask)

Hexadecimal Notation (Hex Mask)

empty mask bit 1 bit 2 bit 3 bit 4 bit 5 bit 6 bit 7 bit 8 full mask

0b00000000 0b00000001 0b00000010 0b00000100 0b00001000 0b00010000 0b00100000 0b01000000 0b10000000 0b11111111

0 1 2 4 8 16 32 64 128 255

0x00 0x01 0x02 0x04 0x08 0x10 0x20 0x40 0x80 0xFF

Figure 6: Binary, decimal and hexadecimal representations of the same byte-sized mask values.

The following code snippet demonstrates how such as set of constants is defined and used in CA-Visual Objects. define define define define ... define define EMPTY_MASK := 0 // 0b0 or 0x00 BIT_1 := 1 // 0b1 or 0x01 BIT_2 := 2 // 0b10 or 0x02 BIT_3 := 4 // 0b100 or 0x04 FULL_MASK := 255 DEFAULT_FLAGS := BIT_1 + BIT_2 // 0b11111111 // 0b11 or 3 decimal

LOCAL Options AS DWORD // Clear the variable Options := EMPTY_MASK // Set a variable to the bit pattern defined as DEFAULT_FLAGS Options := DEFAULT_FLAGS ? Options // 3 Because operating systems use bit flags to provide access to binary settings, the most common application of the bitwise logical operators is to set, clear, test and toggle individual bit flags. Now that the preliminaries have been dispensed, this article considers these four bit-mask operations in detail. Turning On Bit Flags When a bitwise OR is performed on two memory locations, one bit pattern is overlaid on the other, and the resulting bit pattern is the set (one) bits of both operands. Bitwise inclusive OR returns one in all cases except where the corresponding bits of both operands are zero. This property is useful for turning on or setting a specific flag bit, or bit pattern, within a memory location. In the following example, a memory variable, Flag, is assigned one or more flag bits by performing a bitwise OR between that value and the bit mask representing the desired bit positions. define BIT_1 := 0x01 define BIT_2 := 0x02 // Declare and clear a word of memory

A common C usage of masks is ch &= 0377. The value 0377 is 11111111 in binary. This mask leaves the final 8 bits of the variable ch alone and sets the rest to zero. Thus, regardless of whether the original ch is 8, 16, 32 or more bits, the final value is trimmed down to fit into a single byte. This is an important consideration for cross-platform development where the sizes of the primitive data types may vary between platforms. Such operations are also advisable when the operands to the binary bitwise operators are of different-sized data types.

LOCAL Flag := 0 AS WORD // Set flag for bit position 2 Flag := _OR( Flag, BIT_2 ) // Set flag for bit position 1 Flag := _OR( Flag, BIT_1 ) ? Flag // 3 // Clear the byte Flag := 0 // Set flag bits 1 and 2 Flag := _OR( Flag, BIT_1, BIT_2 ) Turning Off Bit Flags Another bit-mask operation is turning off, or zeroing out, one or more bit flags. A bit pattern XORed with itself returns zero. This property is used to turn off one or more flags by setting the specified bit to zero. Assume that a value representing the first two bit positions is stored in the Flags memory variable. The second flag bit can be turned off, or zeroed-out, as follows: define BIT_1 := 0x01 define BIT_2 := 0x02 LOCAL Flags AS DWORD // Set bit 1 and bit 2 Flags := BIT_1 + BIT_2 ? Flags // 3 or 0b11 // Turn off bit 2 Flags := _AND( Flags, _NOT( BIT_2 ) ) // Bit 1 is still set ? Flags // 1 0b1 Since the mask for bit 2, defined as BIT_2, is all zeros except for bit 2, then _NOT( BIT_2 ) is all ones except for bit 2. A one, ANDed with any bit is that bit, so the statement, _AND( Flags, _NOT( BIT_2 ), leaves all the bits other than bit 2 unchanged. For bit 2, a zero AND any bit is zero, so bit 2 is set to zero regardless of its original value. Checking the State of a Bit Flag A bit pattern ANDed with itself returns itself, a property which may be used to determine if a particular bit is set. The bitwise AND operator returns the specified bit pattern if that value is set in the bit mask, otherwise it returns zero. define define define define EMPTY_MASK := 0 BIT_1 := 0x01 BIT_2 := 0x02 DEFAULT_MASK := BIT_1 + BIT_2

LOCAL Flags AS INT // clear the memory location Flags := EMPTY_MASK ? _AND( Flags, BIT_1 ) // 1 Compare _AND() return value with the bit flag definition being tested to obtain a Boolean return value.. s

? _AND( Flags, BIT_1 ) == BIT_1 Checking if a group of bits is set s just as easy.

// TRUE

? AND( Flags, DEFAULT_FLAGS ) == DEFAULT_FLAGS // TRUE Toggling Bit Flags XOR inverts the state of a set of bits, converting all the zeros to ones, and the ones to zeros. In combination with an appropriate bit mask, this property is used to toggle a specific bit or set of bits while leaving all the other bits unchanged. define BIT_1 := 0x01 LOCAL Flags AS BYTE ... Flags := BIT_1 Flags := _XOR( Flags, BIT_1 ) ? Flags // 0 Flags := _XOR( Flags, BIT_1 ) ? Flags // 1

Other uses of XOR may not be immediately obvious, but it is a potent operation (see the sidebar, ZeroSpace Buffer Swapping with XOR).

Zero-Space Buffer Swapping with XOR

A fascinating application of bitwise logical XOR is the exchange of two values without introducing a third memory location. The usual approach uses a third memory variable as temporary storage: temp := integer1 integer1 := integer2 integer2 = temp Zero-space buffer swapping avoids the temporary variable and is commonly used in memory-constrained environments. This technique is simple to implement in CA-Visual Objects. integer1 := _XOR( integer1, integer2 ) integer2 := _XOR( integer2, integer1 ) integer1 := _XOR( integer1, integer2 ) In C, zero-space buffer swapping is extremely important for swapping large character array buffers of identical length. Unlike C/C++, CA-Visual Objects does not require the use of identical length buffers. In CA-Visual Objects 2.x, a similar operation can be performed by casting variables of the pointer to zeroterminated sting data type (PSZ) to DWORD, performing the swap with three successive bitwise XORs, then casting the pointers back to PSZs. The casts simply instruct the compiler to treat a memory location as the specified data type, regardless of its actual declaration. // Swap psz1 and psz2 dw1 := DWORD( _CAST, psz1 ) dw2 := DWORD( _CAST, psz2 ) dw1 := _XOR( dw1, dw2 ) dw2 := _XOR( dw1, dw2 ) dw1 := _XOR( dw1, dw2 ) psz1 := PSZ( _CAST, dw1 ) psz2 := PSZ( _CAST, dw2 ) The above code snippet effectively swaps the pointers of the two zero-terminated strings. Note that the _XOR() runtime function in CA-Visual Objects may be slower than introducing a third variable when swapping integers and small zero-terminated strings. A more compact way can be used to avoid using the temporary DWORD memory variables. psz1 := PSZ( _CAST, _XOR( DWORD(_CAST, psz1), DWORD(_CAST, psz2) ) ) psz2 := PSZ( _CAST, _XOR( DWORD(_CAST, psz1), DWORD(_CAST, psz2) ) ) psz1 := PSZ( _CAST, _XOR( DWORD(_CAST, psz1), DWORD(_CAST, psz2) ) ) See Advanced Example.AEF for more details on the use of casts versus conversions.

Figure 7 summarizes the four typical bit mask operations and provides the CA-Visual Objects prototypes.
Bit-Flag Operation Set a flag or overlay multiple values Zero-out a bit; set it to zero Check if a bit is set Invert a pattern of bits CA-Visual Objects Syntax _OR( <p1>, <p2> ) _XOR( <p1>, _NOT( <p2> ) ) _AND( <p1>, <p2> ) == <p1> _NOT( <p> )

Figure 7: Summary of typical uses of the bitwise logical operators and the CA-Visual Objects syntax.

Practical Bits
One problem with bit mask operations is that the code can be difficult to read and maintain. A simple bitmask operations library serves to hide the implementation details. The following set of functions implements a convenient interface for bit mask operations in CA-Visual Objects 2.x. // A Simple Bit-Mask API // FUNCTION SetBitFlag( dwMask AS DWORD, dwFlag AS DWORD ) AS DWORD PASCAL // Set specified bit flag on RETURN _OR( dwMask, dwFlag ) FUNCTION GetBitFlag( dwMask AS DWORD, dwFlag AS DWORD ) AS LOGIC PASCAL // Get state of the specified bit flag RETURN _AND( dwMask, dwFlag ) == dwMask FUNCTION UnsetBitFlag( dwMask AS DWORD, dwFlag AS DWORD ) AS DWORD PASCAL // Set specified bit flag off // // WARNING: The order of the operands is critical since this function must // apply ones' complement to the mask and not to the flag variable. // RETURN _AND( _NOT( dwMask ), dwFlag ) FUNCTION ToggleBitFlag( dwMask AS DWORD, dwFlag AS DWORD ) AS DWORD PASCAL // Toggle specified bit flag RETURN _XOR( dwMask, dwFlag ) Several sets of such functions could be written to handle all the permissible data types: INT, SHORTINT, LONGINT, WORD and DWORD. Such is the practice in C++ with overloaded functions. CA-Visual Objects provides a different mechanism for achieving the same ends, the USUAL data type. However, because the DWORD type is as large as, or larger than, all the permissible data types, all the functions in the Bit-Mask API work with all the permissible data types. All that is required when using data types other than DWORD is to convert or cast the operands and result to the required data type.

Designer Bits
The view of a memory location as a set of flags is commonly used for directly accessing operating system services. Systems programming applications, such as programming the Microsoft Windows APIs, demand a solid understanding of the bitwise operators and bit-level programming techniques, particularly the four

bit-mask operations. Operating systems, however, are not the only place where bit masking may be usefully employed. Since the techniques may be applied to any memory location, bit mask operations can be extremely useful in streamlining database design and programming, as demonstrated by the following case study. A common application of bit mask operations is the management of security options, which provide different types of user with different levels of access to an application and other system resources. The database storage requirements for a set of security privileges where a user must be granted privileges to access, modify or remove system resources. For simplicity, this case study uses only four privileges. This simple security system's privilege table may be conceptualized as shown in Figure 8.
User ID User 1 User 2 User n View Yes Yes Yes Add No Yes Yes Edit No Yes Yes Delete No No Yes

Figure 8: Logical view of user security profiles table.

This scheme could be used, but it is not very efficient. Using bit mask operations allows for much more flexibility and far greater capabilities. A byte can represent a set of eight Boolean flags. The four bits required for this example, can represent 2^^4-1, or 15, unique security states. Defining the states and rules, and implementing appropriate data access services are key. The security system example requires the definition of only four access rights. To make programming simpler, define the individual bits or bit patterns as follows. define define define define X_VIEW := 1 X_ADD := 2 X_EDIT := 4 X_DELETE := 8

The four access rights are contained in a computer word, or another integer numeric data type that is native to the RDBMS or data storage system being used. The first four bits can represent up to 15 unique access privileges. In the real world, however, not all these privileges make sense. For example, it may not be desirable to grant any other rights if the view right is not granted. Thus, the view right flag acts like a master switch, overriding the other three rights, so the number of meaningful states is less than the theoretical limit. Figure 8 shows the set of meaningful masks that may be handled by a measly four bits.
Bit Position 1 2 3 4 5 6 7 Privilege Definition NO_ACCESS VIEW VIEW + ADD VIEW + EDIT VIEW + DELETE VIEW + ADD + EDIT VIEW + ADD + DELETE VIEW + EDIT + DELETE ALL_ACCESS Binary 0b0000 0b0000 0b0011 0b0101 0b1001 0b0111 0b1011 0b1101 0b1111 Decimal 0 1 3 5 7 9 11 13 15 Hexadecimal 0x00 0x01 0x03 0x05 0x07 0x09 0x0B 0x0D 0x0F

Figure 9: The set of nine privileges are derived from the four primitive rights.

Using bit flags and masks allows the logical view of the security profiles table to be represented by the simple and efficient physical implementation shown in Figure 10. For large systems, this can account for significant reductions in disk space requirements.

10

User ID User 1 User 2 User n

Privilege 1 9 15

Figure 10: Physical view of user security profiles table.

For the implementation of this design, the data access routines use the bitwise logical operations to access the privilege column. In doing so, application programmers require no specialized knowledge as it will appear that they are accessing logical fields containing TRUE or FALSE values.

Conclusion
There are a variety of uses for bitwise operators. This article primarily covered one bit-level programming technique called bit masking operations. In so doing, this article explored CA-Visual Objects 2.0's logical bitwise operators, OR, AND, XOR and NOT, compared their implementations in C/C++ and CA-Visual Objects, and demonstrated how they provide low-level control over and access into the computer s memory. On the practical level, this articled implemented a simple bit mask library using the four bitwise logical operators. It then provided a real-world application of bit-mask programming techniques, a simple access control system, to demonstrate the possible design implications of bit-level programming techniques.

About the Author

Patrick Lazzarotto <plaz@compuserve.com> has been developing business applications for over fourteen years. He has worked on numerous, large-scale development projects using CA-Visual Objects, and was a TechniCon speaker in 1997. Patrick lives in Toronto, Canada.

11