Time for action – starting, stopping, and monitoring MRP
Before starting Redo Apply services, the physical standby database must be in the MOUNT status.
From 11g onwards, the standby database can also be in the OPEN mode. If the redo transport service is in the ARCH mode, the redo will be applied from the archived redo logfiles after being transferred to the standby database. If the redo transport service is in LGWR, the Log network server (LNS) will be reading the redo buffer in SGA and will send redo to Oracle Net Services for transmission to the standby redo logfiles of the standby database using the RFS process. On the standby database, redo will be applied from the standby redo logs.
To execute the following commands, the control file must be a standby control file.
If you execute these commands in a database in the primary mode, Oracle will return an error and ignore the command.
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE;
Start Redo Apply in the foreground.
Connect to the SQLPlus command prompt and issue the following command. If the media recovery is already running, you will run into the error ORA-01153: an incompatible media recovery is active.
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE;
Database altered.
Whenever you issue the preceding command, you can monitor the Redo Apply status from the alert logfile. Managed standby recovery is now active and is not using real-time apply. The SQL session will be active unless you terminate the session by pressing Ctrl + C or kill the session from another active session. Press Ctrl + C to stop Redo Apply.
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE;
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE
*
*
ERROR at line 1:
ORA-16043: Redo apply has been canceled.
ORA-01013: user requested cancel of current operation
Start Redo Apply in the background.
In order to start the Redo Apply service in the background, use the disconnect from session option. This command will return you to the SQL command line once the Redo Apply service is started. Run the following statement on the standby database:
SQL> alter database recover managed standby database disconnect from session;
Database altered.
Check the Redo Apply service status.
From SQL*Plus, you can check whether the Media Recover Process (MRP) is running using the V$MANAGED_STANDBY view:
SQL> SELECT THREAD#,SEQUENCE#,PROCESS,CLIENT_PROCESS,STATUS,BLOCKS FROM V$MANAGED_STANDBY;
THREAD# SEQUENCE# PROCESS CLIENT_P STATUS BLOCKS
---------- ---------- --------- -------- ------------ ----------
1 146 ARCH ARCH CLOSING 1868
1 148 ARCH ARCH CLOSING 6
0 0 ARCH ARCH CONNECTED 0
1 147 ARCH ARCH CLOSING 8
1 149 RFS LGWR IDLE 1
0 0 RFS UNKNOWN IDLE 0
0 0 RFS UNKNOWN IDLE 0
0 0 RFS N/A IDLE 0
1 149 MRP0 N/A APPLYING_LOG 204800
9 rows selected.
From the PROCESS column, you can see that the background process name is MRP0; Media Recovery Process is ACTIVE and the status is APPLYING_LOG, which means that the process is actively applying the archived redo log to the standby database. From the OS, you can monitor the specific background process as follows:
[oracle@oracle-stby ~]$ ps -ef|grep mrp
oracle 5507 1 0 19:26 ? 00:00:02 ora_mrp0_INDIA
From the output, you can simply estimate how many standby instances are running with background recovery. Only one Media Recovery Process can be running per instance.
Also, you can query from v$session.
SQL> select program from v$session where program like '%MRP%';
PROGRAM
-------------------------
oracle@oracle-stby (MRP0)
Stop Redo Apply.
To stop the MRP, issue the following command:
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
Da
tabase altered.
From the alert logfile, you will see the following lines:
Sun Aug 05 21:24:16 2012
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL
Sun Aug 05 21:24:16 2012
MRP0: Background Media Recovery cancelled with status 16037
Errors in file /u02/app/oracle/diag/rdbms/india_un/INDIA/trace/INDIA_mrp0_5507.trc:
ORA-16037: user requested cancel of managed recovery operation
Managed Standby Recovery not using Real Time Apply
Recovery interrupted!
After stopping the MRP, no background process is active and this can be confirmed by using the V$MANAGED_STANDBY or V$SESSION view shown as follows:
SQL> SELECT THREAD#,SEQUENCE#,PROCESS,CLIENT_PROCESS,STATUS,BLOCKS FROM V$MANAGED_STANDBY;
THREAD# SEQUENCE# PROCESS CLIENT_P STATUS BLOCKS
---------- ---------- --------- -------- ------------ ----------
1 146 ARCH ARCH CLOSING 1868
1 148 ARCH ARCH CLOSING 6
0 0 ARCH ARCH CONNECTED 0
1 147 ARCH ARCH CLOSING 8
1 149 RFS LGWR WRITING 1
0 0 RFS UNKNOWN IDLE 0
0 0 RFS UNKNOWN IDLE 0
0 0 RFS N/A IDLE 0
8 rows selected.
SQL> select program from v$session where program like '%MRP%';
no rows selected
Start real-time apply.
To start Redo Apply in real-time apply mode, you must use the USING CURRENT LOGFILE option as follows:
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT FROM SESSION;
Database altered.
From the standby alert logfile, you will see the following lines:
Sun Aug 05 15:31:21 2012
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT FROM SESSION
Attempt to start background Managed Standby Recovery process (INDIA)
Sun Aug 05 15:31:21 2012
========================================
Time for action – closing a gap with an RMAN incremental backup
Let's see all the required steps to practice this recovery operation:
In this practice, assume that there are missing archived logs (gap) in the standby database, and we're not able to restore these archived logs. We'll synchronize Data Guard using the RMAN incremental backup. To represent this situation, execute the DEFER command to defer the log destination in the primary database, and execute the following operation that will generate redo in the primary database:
SQL> ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_2 = 'DEFER';
Now we have a standby database behind the primary database, and we'll use RMAN to reflect the primary database's changes to the standby database. Stop Redo Apply in the standby database:
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
Query the current system change number (SCN) of the standby database that will be used as the limit for an incremental backup of the primary database. Run the following statement on the standby database:
SQL> SELECT MIN(FHSCN) FROM X$KCVFH;
MIN(FHSCN)
----------------
20606344
Run an RMAN incremental backup of the primary database by using the obtained SCN value.
Tip
This backup job will check all the blocks of the primary database and back up the blocks that have a higher SCN. So even if the backup size is small, it may take a long time.
RMAN> BACKUP INCREMENTAL FROM SCN 20606344 DATABASE FORMAT '/tmp/Standby_Inc_%U' tag 'STANDBY_INC';
Starting backup at 20-DEC-12
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=165 device type=DISK
backup will be obsolete on date 27-DEC-12
archived logs will not be kept or backed up
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00001 name=/u01/app/oracle2/datafile/ORCL/system01.dbf
...
input datafile file number=00007 name=/u01/app/oracle2/datafile/ORCL/system03.dbf
channel ORA_DISK_1: starting piece 1 at 20-DEC-12
channel ORA_DISK_1: finished piece 1 at 20-DEC-12
piece handle=/tmp/Standby_Inc_03nt9u0v_1_1 tag=STANDBY_INC comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:01:15
using channel ORA_DISK_1
including current control file in backup set
channel ORA_DISK_1: starting piece 1 at 20-DEC-12
channel ORA_DISK_1: finished piece 1 at 20-DEC-12
piece handle=/tmp/Standby_Inc_04nt9u3a_1_1 tag=STANDBY_INC comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 20-DEC-12
Copy the backup files from the primary site to the standby site with FTP or SCP.
scp /tmp/Standby_Inc_* standbyhost:/tmp/
Register the backup files to the standby database control file with the RMAN CATALOG command, so that we'll be able to recover the standby database using these backup files:
RMAN> CATALOG START WITH '/tmp/Standby_Inc';
using target database control file instead of recovery catalog
searching for all files that match the pattern /tmp/Standby_Inc
List of Files Unknown to the Database
=====================================
File Name: /tmp/Standby_Inc_03nt9u0v_1_1
File Name: /tmp/Standby_Inc_04nt9u3a_1_1
Do you really want to catalog the above files (enter YES or NO)? YES
cataloging files...
cataloging done
List of Cataloged Files
=======================
File Name: /tmp/Standby_Inc_03nt9u0v_1_1
File Name: /tmp/Standby_Inc_04nt9u3a_1_1
Recover the standby database with the RMAN RECOVER statement. The Recovery operation will use the incremental backup by default as we have already registered the backup files:
RMAN> RECOVER DATABASE NOREDO;
Starting recover at 20-DEC-12
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=1237 device type=DISK
channel ORA_DISK_1: starting incremental datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
destination for restore of datafile 00001: /u01/app/oracle2/datafile/INDIAPS/system01.dbf
...
destination for restore of datafile 00007: /u01/app/oracle2/datafile/INDIAPS/system03.dbf
channel ORA_DISK_1: reading from backup piece /tmp/Standby_Inc_03nt9u0v_1_1
channel ORA_DISK_1: piece handle=/tmp/Standby_Inc_03nt9u0v_1_1 tag=STANDBY_INC
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
Finished recover at 20-DEC-12
In this step, we'll create a new standby control file in the primary database and open the standby database using this new control file. We've performed this process at the beginning of this chapter, so we won't be explaining it again; only the statements are given as follows:
In the primary database you will see the following command lines:
RMAN> BACKUP CURRENT CONTROLFILE FOR STANDBY FORMAT '/tmp/Standby_CTRL.bck';
scp /tmp/Standby_CTRL.bck standbyhost:/tmp/
In the standby database you will see the following command lines:
RMAN> SHUTDOWN;
RMAN> STARTUP NOMOUNT;
RMAN> RESTORE STANDBY CONTROLFILE FROM '/tmp/Standby_CTRL.bck';
RMAN> SHUTDOWN;
RMAN> STARTUP MOUNT;
If OMF is being used, execute the following commands:
RMAN> CATALOG START WITH '+DATA/mystd/datafile/';
RMAN> SWITCH DATABASE TO COPY;
If new datafiles were added during the time when Data Guard had been stopped, we will need to copy and register the newly created files to the standby system, as they were not included in the incremental backup set.
We will determine if any files have been added to the primary database, as the standby current SCN will run the following query:
SQL>SELECT FILE#, NAME FROM V$DATAFILE WHERE CREATION_CHANGE# > 20606344;
If the flashback database is ON in the standby database, turn it off and on again:
SQL> ALTER DATABASE FLASHBACK OFF;
SQL> ALTER DATABASE FLASHBACK ON;
Clear all the standby redo log groups in the standby database:
SQL> ALTER DATABASE CLEAR LOGFILE GROUP 4;
SQL> ALTER DATABASE CLEAR LOGFILE GROUP 5;
SQL> ALTER DATABASE CLEAR LOGFILE GROUP 6;
SQL> ALTER DATABASE CLEAR LOGFILE GROUP 7;
Start Redo Apply in the standby database:
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT FROM SESSION;
========================
Time for action – resolving UNNAMED datafile errors
How to you check the datafiles that needs to be recovered ?
out put is from STandby database ?
SELECT * FROM V$RECOVER_FILE WHERE ERROR LIKE '%MISSING%';
SQL> SELECT * FROM V$RECOVER_FILE WHERE ERROR LIKE '%MISSING%';
FILE# ONLINE ONLINE_ ERROR CHANGE# TIME
---------- ------- ------- ----------------- ---------- ----------
10 ONLINE ONLINE FILE MISSING 0
2.Identify datafile 10 in the primary database
SQL> SELECT FILE#,NAME FROM V$DATAFILE WHERE FILE#=10;
FILE# NAME
---------- -----------------------------------------------
536 /u01/app/oracle2/datafile/ORCL/users03.dbf
3.Identify the dummy filename created in the standby database:
SQL> SELECT FILE#,NAME FROM V$DATAFILE WHERE FILE#=10;
FILE# NAME
---------- -------------------------------------------------------
536 /u01/app/oracle2/product/11.2.0/dbhome_1/dbs/UNNAMED00010
4.If the reason for the creation of the UNNAMED file is disk capacity or a nonexistent path, fix the issue by creating the datafile in its original place.
5.Set STANDBY_FILE_MANAGEMENT to MANUAL:
SQL> ALTER SYSTEM SET STANDBY_FILE_MANAGEMENT=MANUAL;
System altered.
6.Create the datafile in its original place with the ALTER DATABASE CREATE DATAFILE statement:
SQL> ALTER DATABASE CREATE DATAFILE '/u01/app/oracle2/product/11.2.0/dbhome_1/dbs/UNNAMED00010' AS '/u01/app/oracle2/datafile/ORCL/users03.dbf';
Database altered.
7.Set STANDBY_FILE_MANAGEMENT to AUTO and start Redo Apply:
SQL> ALTER SYSTEM SET STANDBY_FILE_MANAGEMENT=AUTO SCOPE=BOTH;
System altered.
SQL> SHOW PARAMETER STANDBY_FILE_MANAGEMENT
NAME TYPE VALUE
----------------------------------- ----------- ------------------
standby_file_management string AUTO
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT FROM SESSION;
Database altered.
8.Check the standby database's processes, or the alert log file, to monitor Redo Apply:
SQL> SELECT PROCESS, STATUS, THREAD#, SEQUENCE#, BLOCK#, BLOCKS FROM V$MANAGED_STANDBY;
=======================
Fixing NOLOGGING changes on the standby database
It's possible to limit redo generation for specific operations on Oracle databases, which provide higher performance. These operations include bulk inserts, creation of tables as select operations, and index creations. When we work using the NOLOGGING clause, redo will not include all the changes to data on the related segments. This means if we perform a restore/recovery of the related datafile, or of the whole database after the NOLOGGING operations, it'll not be possible to recover the data created with the NOLOGGING option.
The same problem exists with Data Guard. When the NOLOGGING operation is executed in the primary database, Data Guard is not able to reflect all the data changes in the standby database. In this case, when we activate a standby database or open it in the read-only mode, we'll see the following error messages:
ORA-01578: ORACLE data block corrupted (file # 1, block # 2521)
ORA-01110: data file 1: '/u01/app/oracle2/datafile/INDIAPS/system01.dbf'
ORA-26040: Data block was loaded using the NOLOGGING option
For this reason, Data Guard installation requires putting the primary database in the FORCE LOGGING mode before starting redo transport between the primary and standby database. The FORCE LOGGING mode guarantees the writing of redo records even if the NOLOGGING clause was specified in the SQL statements. The default mode of an Oracle database is not FORCE LOGGING, so we need to put the database in this mode using the following statement:
SQL> ALTER DATABASE FORCE LOGGING;
In this section, we'll assume that the primary database is not in the FORCE LOGGING mode, and some NOLOGGING changes were made in the primary database. One method to fix this situation in the standby database is restoring the affected datafiles from backups taken from the primary database after the NOLOGGING operation. However, in this method we have to work with backup files that are most likely much bigger in size than the amount of data that needs to be recovered. A method that uses the RMAN BACKUP INCREMENTAL FROM SCN statement is more efficient because the backup files will include only the changes from the beginning of the NOLOGGING operation.
We'll now see two scenarios. We'll use the BACKUP INCREMENTAL FROM SCN statement for an incremental datafile backup in the first scenario, and use the same statement for an incremental database backup in the second one. For a small number of affected datafiles and relatively less affected data, choose the first scenario. However, if the number of affected datafiles and amount of data are high, use the second scenario that takes an incremental backup of the whole database.
Time for action – fixing NOLOGGING changes on a standby database with incremental datafile backups
As a prerequisite for this exercise, first put the primary database in the no-force logging mode using the ALTER DATABASE NO FORCE LOGGING statement. Then perform some DML operations in the primary database using the NOLOGGING clause so that we can fix the issue in the standby database with the following steps:
1.Run the following query to identify the datafiles that are affected by NOLOGGING changes:
SQL> SELECT FILE#, FIRST_NONLOGGED_SCN FROM V$DATAFILE WHERE FIRST_NONLOGGED_SCN > 0;
FILE# FIRST_NONLOGGED_SCN
---------- -------------------
4 20606544
2.First we need to put the affected datafiles in the OFFLINE state in the standby database. For this purpose, stop Redo Apply in the standby database, execute the ALTER DATABASE DATAFILE ... OFFLINE statement, and start Redo Apply again:
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
SQL> ALTER DATABASE DATAFILE 4 OFFLINE FOR DROP;
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT;
3.Now we'll take incremental backups of the related datafiles by using the FROM SCN keyword. SCN values will be the output of the execution of the queries in the first step. Connect to the primary database as an RMAN target and execute the following RMAN BACKUP statements:
RMAN> BACKUP INCREMENTAL FROM SCN 20606544 DATAFILE 4 FORMAT '/data/Dbf_inc_%U' TAG 'FOR STANDBY';
4.Copy the backup files from the primary site to the standby site with FTP or SCP:
scp /data/Dbf_inc_* standbyhost:/data/
5.Connect to the physical standby database as the RMAN target and catalog the copied backup files to the control file with the RMAN CATALOG command:
RMAN> CATALOG START WITH '/data/Dbf_inc_';
6.In order to put the affected datafiles in the ONLINE state, stop Redo Apply on the standby database, and run the ALTER DATABASE DATAFILE ... ONLINE statement:
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
SQL> ALTER DATABASE DATAFILE 4 ONLINE;
7.Recover the datafiles by connecting the standby database as the RMAN target. RMAN will use the incremental backup automatically because those files were registered to the control file previously:
RMAN> RECOVER DATAFILE 4 NOREDO;
8.Now run the query from the first step again to ensure that there're no more datafiles with the NOLOGGING changes:
SQL> SELECT FILE#, FIRST_NONLOGGED_SCN FROM V$DATAFILE WHERE FIRST_NONLOGGED_SCN > 0;
9.Start Redo Apply on the standby database:
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT;
=================
Cloning Database .
=================
1)First Check do we have enough space or not in target space.
2)Please ensure that no duplicate datafiles are present in database.
If there are duplicate datafiles in the database,make sure to map them to different mountpoints in the restore script .
If restore fails due to duplicate file issues(datafiles switch not happends),then only restore the failed datafiles(get the information from log),
then we need to switching for all datafiles .
3) Make sure that the once restore and recovery of the database is completed, please rename the redo logfiles as per cloned server
mountpoint details and permission of the redo logfiles should be oracle:dba
Now actual steps
1) Backup the database with controlfile to any mountpoint.
2) copy to source database Pfile to Target database and change the Database name.
3) Create a restore script .
How restore script look like ?
it contains..
run {
allocate channel c1 type disk;
allocate channel c2 type disk;
SET NEWNAME FOR DATAFILE 1 to '/glerpq02/data/pnecqa2/data01/pnecqa2/abc.dbf';
SET NEWNAME FOR DATAFILE 2 to '/glerpq02/data/pnecqa2/data01/pnecqa2/bcd.dbf';
…………….
………..
restore database;
switch datafile all;
release channel c1;
release channel c2;
}
Drop the database if it is already existing. First shutdown the database,listener and then use rm -rf command from OS level
to remove the datafiles,tempfiles,controlfile,logfiles present in the mount points of existing database.
Use the following commands to check and remove the datafiles,tempfiles,controlfile,logfiles present in the mount points of existing database.:---
SQL> select name from v$datafile; (To view datafiles)
SQL> select name from v$controlfile; (To view controlfiles)
SQL> select member from v$logfile; (To view logfiles)
SQL> select name from v$tempfile; (To view tempfiles)
Create a restore shell script that would call this restore script.
rman target=/ cmdfile=restore.rcv log=restore.log
CLoning starts now.
1) Edit pfile
2) startup nomount
go to rman prompt
3) rman target /
Restore the controlfile from the backup piece.
RMAN> restore controlfile from 'fullbackup_ctl<along with full path>';
8) Mount the database
9) Now start the restore script.
Now restore the datafiles to new locations and recover
$ nohup restore.sh &
RMAN> run
{
allocate channel c1 type disk;
allocate channel c2 type disk;
allocate channel c3 type disk;
allocate channel c4 type disk;
recover database;
release channel c1;
release channel c2;
release channel c3;
release channel c4;
}
12) Once database is recovered .
please rename the redo log files for the database as per existing mountpoints.
SQL>select member from v$logfile ;
SQL> alter database rename file ‘source path’ to ‘destination path’;
13) open the database using resetlogs
14) Shutdown the database
15) Mount the database
SQL> startup mount;
16) Invoke nid utility and allow to complete
nid target=/ dbname=clone
Note:--- If we are performing manual recovery we by using backup controlfile,
First we should have to set the clone database archive destination to the mount point
in which archives logs resides and also change the archive log format of clone server to the archive log format of target server.
RECOVER BY USING BACKUP CONTROLFILE COMMAND:-----
SQl> recover database using backup controlfile;
Specify auto if it prompts for auto | manual | cancel
RECOVER CANCEL BY USING BACKUP CONTROLFILE COMMAND:-----
SQl> recover database using backup controlfile until cancel;
Specify cancel if it prompts for auto | manual | cancel