5
Oracle Instance Architecture for the Parallel Server

[Architecture] is music in space, as it were a frozen music...

- Schelling, Philosophie der Kunst

This chapter explains features of Oracle multi-instance architecture which differ from an Oracle server in exclusive mode.

Overview

Each Oracle instance in a parallel server architecture has its own

system global area (SGA)
background processes
ORACLE_SID
set of redo logs

All instances in a parallel server database share

the same set of data files
the same set of control files

The Oracle Parallel Server (OPS) instance contains

an additional PCM lock area in its SGA to coordinate the use of shared resources (also known as lock elements)
the Integrated Distributed Lock Manager component, an area for global locks and resources. This area was formerly part of the external distributed lock manager (DLM).
additional background processes LCKn to coordinate the locking of shared resources among the multiple instances in a parallel server
additional background processes LMON and LMD0 to manage global locks and resources

Basic elements of the Oracle Parallel Server are illustrated in Figure 5-1. DBWR processes are shown writing data, users reading data.

Figure 5-1 Basic Elements of Oracle Parallel Server

See Also: "Memory Structures and Processes" in Oracle8 Concepts.

Characteristics of OPS Multi-instance Architecture

Characteristics of an Oracle parallel server can be summarized as follows:

An Oracle instance can be started on one or more nodes in the network.
Each instance has a separate System Global Area (SGA) and set of background processes.
All instances share the same datafiles and control file.
Each instance has its own set of redo log files.
Note: The redo logs must be accessible to all instances in case of instance failure. On some MPP platforms, a redo server exists so that only one set of redo logs is necessary for the whole OPS system.
Archived logs are private, but must be accessible to all instances for media recovery.
All instances can execute transactions concurrently against the same database, and each instance can have multiple users executing transactions.
Row level locking is preserved.

A parallel server is administered in the same manner as a non-parallel server, except that you must connect to a particular instance to perform administration.

Applications that access the database can run on the same nodes as instances of a parallel server or on separate nodes, using the client-server architecture. A parallel server can be part of a distributed database system. Distributed transactions access the data in a remote database in the same manner, regardless of whether the datafiles are owned by an Oracle Server (in exclusive mode) or a parallel server (in exclusive or shared mode).

Other non-Oracle processes can run on each node of the system, or you can dedicate the entire system or part of the system to Oracle. For example, a parallel server and its applications might occupy three nodes of a five-node configuration, while the other two nodes are used for non-Oracle applications.

System Global Area

Each instance of a parallel server has its own System Global Area (SGA). The SGA includes the following memory structures:

buffer cache for data blocks
dictionary cache for data dictionary information
redo log buffer for redo entries
shared pool containing the shared SQL and shared PL/SQL areas
instance lock area (only in a parallel server)

Data sharing between SGAs in a parallel server is controlled by parallel cache management, which uses parallel cache management (PCM) locks.

A data block can be present in several SGAs at the same time. PCM locks ensure that the database buffer cache is kept consistent for all the instances. It thus ensures readability by one instance of changes made by other instances.

Each instance has a shared pool that can only be used by the user applications connected to that instance. If the same SQL statement is submitted by different applications using the same instance, it is parsed and stored once in that instance's SGA. If that same SQL statement is also submitted by an application on a different instance, then this different instance also parses and stores the statement.

Background Processes and LCKn

Each instance in a parallel server has its own set of background processes, which are identical to the background processes of a single server in exclusive mode. The DBWR, LGWR, PMON, and SMON processes are present for every instance; the optional processes, ARCH, CKPT, Dnnn and RECO, can be enabled by setting the appropriate initialization parameters. In addition to the standard background processes, each instance of a parallel server has at least one lock process, LCK0. Additional lock processes can be enabled if necessary.

In OPS the Integrated Distributed Lock Manager also uses the LMON and LMD0 processes. LMON is used to manage instance and process deaths and associated recovery for the Integrated DLM. In particular, LMON handles the part of recovery that is associated with the global locks. The LMD processes are used to handle remote lock requests (those which originate from other instances).

The Lock process (LCKn) manages the locks used by an instance and coordinates requests for those locks by other instances. Additional lock processes, LCK1 through LCK9, are available for systems that require exceptionally high throughput of instance lock requests. The single lock process per instance, LCK0, is usually sufficient for most systems.

All instances in a parallel server must have the same number of lock processes. Lock processes use the Integrated DLM to coordinate the buffer caches of the different SGAs in a parallel server.

When an instance fails in shared mode, another instance's SMON detects the failure and recovers for the failed instance. The lock processes of the instance doing the recovery clean up any outstanding PCM locks for the failed instance.

All the lock processes for one instance are functionally equivalent. Typically, one lock process is sufficient. Occasionally, more than one lock process may maximize throughput of locking requests. Although lock requests are asynchronous, each request is blocked until the lock process knows if the lock can be granted immediately. Since multiple lock processes are functionally equivalent, this manual refers only to the lock process.

See Also: "The LCKn Processes" on page 7-6
"GC_* Initialization Parameters" on page 9-13

Configuration Guidelines for Oracle Parallel Server

When setting up an Oracle Parallel Server environment, observe the guidelines presented in Table 5-1:

Table 5-1 Parallel Server Configuration Guidelines

Configuration Issue	Guidelines
Version	Ensure that the same Oracle version exists on all the nodes.
Links	UNIX soft or hard inks ("aliases") to executables are not recommended for a parallel server. If the single node containing the executables fails, none of the nodes will be able to operate.
Initialization parameters	Keep in a single file the initialization parameters which should be identical across all instances in a parallel server. Include this file in the individual initialization files of the different instances using the IFILE option.
Control files	Must be accessible from all instances.
Data files	Must be accessible from all instances.
Log files	Must be located on the same set of disks as control files and data files. Although the redo log files are independent per instance, each of the log files must still be accessible by all the instances so that recovery is provided for.
NFS	You can use NFS to enable access to Oracle executables, but not access to database files or log files. Note that if you are using NFS, the serving node is a single point of failure.
Archived redo log files	Must be accessible from all instances.

5Oracle Instance Architecture for the Parallel Server