It's All About ORACLE

Oracle - The number one Database Management System. Hope this Blog will teach a lot about oracle.

Refreshing a Materialized View

Historically, one of the problems of using summary tables has been the initial loading and subsequent updating of the summary. These issues are now addressed because summary management provides mechanisms to:
• Fully refresh the data.
• Perform a fast refresh, that is add/merge only the changes.
• Automatically update a materialized view whenever changes are made.
Therefore the DBA must consider how much time is required to create and maintain each materialized view and balance that against the performance gains achieved by using this materialized view.

When you define a materialized view, you can specify three refresh options: how to refresh, what type of refresh, and can trusted constraints be used. If unspecified, the defaults are assumed as ON DEMAND, FORCE, and ENFORCED constraints respectively.

The two refresh execution modes are ON COMMIT and ON DEMAND. 
Table 1 Refresh Modes
Refresh ModeDescription
ON COMMITRefresh occurs automatically when a transaction that modified one of the materialized view's detail tables commits. This can be specified as long as the materialized view is fast refreshable (in other words, not complex). The ON COMMIT privilege is necessary to use this mode
ON DEMANDRefresh occurs when a user manually executes one of the available refresh procedures contained in the DBMS_MVIEW package (REFRESH, REFRESH_ALL_MVIEWS, REFRESH_DEPENDENT)

When a materialized view is maintained using the ON COMMIT method, the time required to complete the commit may be slightly longer than usual. This is because the refresh operation is performed as part of the commit process. Therefore this method may not be suitable if many users are concurrently changing the tables upon which the materialized view is based.

If you anticipate performing insert, update or delete operations on tables referenced by a materialized view concurrently with the refresh of that materialized view, and that materialized view includes joins and aggregation, Oracle recommends you use ON COMMIT fast refresh rather than ON DEMAND fast refresh.

If a materialized view fails during refresh at COMMIT time, you must explicitly invoke the refresh procedure using the DBMS_MVIEW package after addressing the errors specified in the trace files. Until this is done, the materialized view will no longer be refreshed automatically at commit time.

You can specify how you want your materialized views to be refreshed from the detail tables by selecting one of four options: COMPLETE, FAST, FORCE, and NEVER

Table 2 Refresh Options

Refresh OptionDescription
COMPLETERefreshes by recalculating the materialized view's defining query.
FASTApplies incremental changes to refresh the materialized view using the information logged in the materialized view logs, or from a SQL*Loader direct-path or a partition maintenance operation.
FORCEApplies FAST refresh if possible; otherwise, it applies COMPLETE refresh.
NEVERIndicates that the materialized view will not be refreshed with refresh mechanisms.

Whether the fast refresh option is available depends upon the type of materialized view. You can call the procedure DBMS_MVIEW.EXPLAIN_MVIEW to determine whether fast refresh is possible.

You can also specify if it is acceptable to use trusted constraints and REWRITE_INTEGRITY = TRUSTED during refresh. Any nonvalidated RELY constraint is a trusted constraint. For example, nonvalidated foreign key/primary key relationships, functional dependencies defined in dimensions or a materialized view in the UNKNOWN state. If query rewrite is enabled during refresh, these can improve the performance of refresh by enabling more performant query rewrites. Any materialized view that can uses TRUSTED constraints for refresh is left in a state of trusted freshness (the UNKNOWN state) after refresh.

This is reflected in the column STALENESS in the view USER_MVIEWS. The column UNKNOWN_TRUSTED_FD in the same view is also set to Y, which means yes.

You can define this property of the materialized either during create time by specifying REFRESH USING TRUSTED [ENFORCED] CONSTRAINTS or by using ALTER MATERIALIZED VIEW DDL.

Table 3 Constraints
Constraints to UseDescription
TRUSTED CONSTRAINTS
Refresh can use trusted constraints and REWRITE_INTEGRITY = TRUSTED during refresh.This allows use of non-validated RELY constraints and rewrite against materialized views in UNKNOWN orFRESH state during refresh.
ENFORCED CONSTRAINTS
Refresh can use validated constraints and REFRESH_INTEGRITY=ENFORCED during refresh. This allows use of only validated, enforced constraints and rewrite against materialized views inFRESH state during refresh.

These operations may be performed: 
• On demand refreshing by: 
 Specific materialized views (DBMS_MVIEW.REFRESH) 
 Those materialized views dependent on a table (DBMS_MVIEW.REFRESH_DEPENDENT) 
 All materialized views (DBMS_MVIEW.REFRESH_ALL_MVIEWS) 
• on commit, whenever the tables on which the materialized view is defined are changed 

The ON DEMAND refresh is achieved by calling one of the procedures listed above, thus giving the DBA total control over when a materialized view is update. 

If the ON COMMIT refresh method is chosen, whenever a materialized view is affected by changes made to the source data, the materialized view will automatically be updated to reflect this data. However, it should be remembered that this update to the materialized view occurs as part of the commit processing in the transaction where the changes to the base table are made. Therefore, the commit will take slightly longer, because changes are being made to both the original table and then any materialized view whose definition includes that table.

Complete Refresh 

When a complete refresh of a materialized view occurs, all data is removed and then it is reloaded. Depending on the size of the materialized view, this could be a time consuming operation. Complete refresh is a good technique to use when: 
• the number of new rows to be inserted is more than 50% of the cardinality of the tables on which the materialized view is based 
• there is no index on the materialized view that is usable for merging 
• the time required to perform a fast refresh is longer than a complete refresh

Fast Refresh 

Some materialized views could be very large and the time required to regularly perform a complete refresh may not be available. The alternative is a fast refresh where only the changes to the detail table are applied against the materialized view. New data loaded into any table in the warehouse is identified and any materialized view referencing that table is automatically updated with the new data. 

In order to perform a fast refresh operation, changes made to the data must be recorded and this is achieved in one of two ways. If your data is only ever inserted into the database using SQL*Loader direct path, then the refresh mechanism will detect this and identify the new data to be loaded. However, many data changes will occur via the SQL commands, INSERT, UPDATE and DELETE. In this case a MATERIALIZED VIEW LOG is required on each of the tables on which the materialized views are based. 

Only one log is required per table and the materialized view log is on the table, not the materialized view. Therefore if you only had 6 tables in your database that were changing, you would only need 6 materialized view logs. But you could have any number of materialized views using those logs. 

It should be noted that not all materialized views are fast refreshable and confirmation of whether it is possible can be obtained by calling the procedure DBMS_MVIEW.EXPLAIN_MVIEW. This procedure will also advise what needs to be done to the materialized view to make it fast refreshable. The procedure TUNE_MVIEW can then be called to generate a script, which will show, if it is possible, how to make the materialized view fast refreshable. 

Syntax for creating Materialized View:
CREATE MATERIALIZED VIEW
  [ schema. ]materialized_view
  [ OF [ schema. ]object_type ]
  [ (scoped_table_ref_constraint) ]
  { ON PREBUILT TABLE
    [ { WITH | WITHOUT } REDUCED PRECISION ]
  | physical_properties materialized_view_props
  }
  [ USING INDEX
    [ physical_attributes_clause
    | TABLESPACE tablespace
    ]
      [ physical_attributes_clause
      | TABLESPACE tablespace
      ]...
  | USING NO INDEX
  ]
  [ create_mv_refresh ]
  [ FOR UPDATE ]
  [ { DISABLE | ENABLE }
QUERY REWRITE
  ]
AS subquery ;

Syntax for REFRESH option:


{ REFRESH
  { { FAST | COMPLETE | FORCE }
  | ON { DEMAND | COMMIT }
  | { START WITH | NEXT } date
  | WITH { PRIMARY KEY | ROWID }
  | USING
       { DEFAULT [ MASTER | LOCAL ]
ROLLBACK SEGMENT
       | [ MASTER | LOCAL ]
ROLLBACK SEGMENT rollback_segment
       }
         [ DEFAULT [ MASTER | LOCAL ]
ROLLBACK SEGMENT
         | [ MASTER | LOCAL ]
ROLLBACK SEGMENT rollback_segment
         ]...
  | USING
       { ENFORCED | TRUSTED }
CONSTRAINTS
  }
    [ { FAST | COMPLETE | FORCE }
    | ON { DEMAND | COMMIT }
    | { START WITH | NEXT } date
    | WITH { PRIMARY KEY | ROWID }
    | USING
         { DEFAULT [ MASTER | LOCAL ]
ROLLBACK SEGMENT
         | [ MASTER | LOCAL ]
ROLLBACK SEGMENT rollback_segment
         }
           [ DEFAULT [ MASTER | LOCAL ]
ROLLBACK SEGMENT
           | [ MASTER | LOCAL ]
ROLLBACK SEGMENT rollback_segment
           ]...
    | USING
         { ENFORCED | TRUSTED }
CONSTRAINTS
    ]...
| NEVER REFRESH
}

Use the refresh clause to specify the default methods, modes, and times for the database to refresh the materialized view. If the master tables of a materialized view are modified, then the data in the materialized view must be updated to make the materialized view accurately reflect the data currently in its master tables. This clause lets you schedule the times and specify the method and mode for the database to refresh the materialized view.

FAST Clause
Specify FAST to indicate the incremental refresh method, which performs the refresh according to the changes that have occurred to the master tables. The changes for conventional DML changes are stored in the materialized view log associated with the master table.The changes for direct-path INSERT operations are stored in the direct loader log.


If you specify REFRESH FAST, then the CREATE statement will fail unless materialized view logs already exist for the materialized view master tables. Oracle Database creates the direct loader log automatically when a direct-path INSERT takes place. No user intervention is needed.

For both conventional DML changes and for direct-path INSERT operations, other conditions may restrict the eligibility of a materialized view for fast refresh.
Materialized views are not eligible for fast refresh if the defining query contains an analytic function.

COMPLETE Clause
Specify COMPLETE to indicate the complete refresh method, which is implemented by executing the defining query of the materialized view. If you request a complete refresh, then Oracle Database performs a complete refresh even if a fast refresh is possible.


FORCE Clause
Specify FORCE to indicate that when a refresh occurs, Oracle Database will perform a fast refresh if one is possible or a complete refresh otherwise. If you do not specify a refresh method (FAST, COMPLETE, or FORCE), then FORCE is the default.

ON COMMIT Clause
Specify ON COMMIT to indicate that a fast refresh is to occur whenever the database commits a transaction that operates on a master table of the materialized view. This clause may increase the time taken to complete the commit, because the database performs the refresh operation as part of the commit process.


Restriction on Refreshing ON COMMIT
This clause is not supported for materialized views containing object types.

ON DEMAND Clause
Specify ON DEMAND to indicate that the materialized view will be refreshed on demand by calling one of the three DBMS_MVIEW refresh procedures. If you omit both ON COMMIT and ON DEMAND, ON DEMAND is the default.

If you specify ON COMMIT or ON DEMAND, then you cannot also specify START WITH or NEXT.

START WITH Clause
Specify a datetime expression for the first automatic refresh time.

NEXT Clause

Specify a datetime expression for calculating the interval between automatic refreshes.
Both the START WITH and NEXT values must evaluate to a time in the future. If you omit the START WITH value, then the database determines the first automatic refresh time by evaluating the NEXT expression with respect to the creation time of the materialized view. If you specify a START WITH value but omit the NEXT value, then the database refreshes the materialized view only once. If you omit both the START WITH and NEXT values, or if you omit the create_mv_refresh entirely, then the database does not automatically refresh the materialized view

WITH PRIMARY KEY Clause 
Specify WITH PRIMARY KEY to create a primary key materialized view. This is the default and should be used in all cases except those described for WITH ROWID. Primary key materialized views allow materialized view master tables to be reorganized without affecting the eligibility of the materialized view for fast refresh. The master table must contain an enabled primary key constraint.

WITH ROWID Clause
Specify WITH ROWID to create a rowid materialized view. Rowid materialized views are useful if the materialized view does not include all primary key columns of the master tables. Rowid materialized views must be based on a single table and cannot contain any of the following:
  • Distinct or aggregate functions
  • GROUP BY or CONNECT BY clauses
  • Subqueries
  • Joins
  • Set operations
Rowid materialized views are not eligible for fast refresh after a master table reorganization until a complete refresh has been performed.

USING ROLLBACK SEGMENT Clause 
This clause is not valid if your database is in automatic undo mode, because in that mode Oracle Database uses undo tablespaces instead of rollback segments. Oracle strongly recommends that you use automatic undo mode. This clause is supported for backward compatibility with replication environments containing older versions of Oracle Database that still use rollback segments.
For rollback_segment, specify the remote rollback segment to be used during materialized view refresh.

MASTER
MASTER specifies the remote rollback segment to be used at the remote master site for the individual materialized view.

LOCAL

LOCAL specifies the remote rollback segment to be used for the local refresh group that contains the materialized view. This is the default.

If you omit rollback_segment, then the database automatically chooses the rollback segment to be used. One master rollback segment is stored for each materialized view and is validated during materialized view creation and refresh. If the materialized view is complex, then the database ignores any master rollback segment you specify.

USING ... CONSTRAINTS Clause 
The USING ... CONSTRAINTS clause lets Oracle Database choose more rewrite options during the refresh operation, resulting in more efficient refresh execution. The clause lets Oracle Database use unenforced constraints, such as dimension relationships or constraints in the RELY state, rather than relying only on enforced constraints during the refresh operation.

NEVER REFRESH Clause
Specify NEVER REFRESH to prevent the materialized view from being refreshed with any Oracle Database refresh mechanism or packaged procedure. Oracle Database will ignore any REFRESH statement on the materialized view issued from such a procedure. To reverse this clause, you must issue an ALTER MATERIALIZED VIEW ... REFRESH statement.


Examples

Creating a Materialized View Log: Examples
The following statement creates a materialized view log on the oe.customers table that specifies physical and storage characteristics:
CREATE MATERIALIZED VIEW LOG ON customers 
   PCTFREE 5 
   TABLESPACE example 
   STORAGE (INITIAL 10K NEXT 10K); 

This materialized view log supports fast refresh for primary key materialized views only. 

The following statement creates another version of the materialized view log with the ROWID clause, which enables fast refresh for more types of materialized views:
CREATE MATERIALIZED VIEW LOG ON customers WITH PRIMARY KEY, ROWID; 

This materialized view log makes fast refresh possible for rowid materialized views and for materialized join views. To provide for fast refresh of materialized aggregate views, you must also specify the SEQUENCE and INCLUDING NEW VALUES clauses, as shown in the next statement.

Specifying Filter Columns for Materialized View Logs: Example
The following statement creates a materialized view log on the sh.sales table and is used in "Creating Materialized Aggregate Views: Example". It specifies as filter columns all of the columns of the table referenced in that materialized view.
CREATE MATERIALIZED VIEW LOG ON sales 
   WITH ROWID, SEQUENCE(amount_sold, time_id, prod_id)
   INCLUDNG NEW VALUES; 

Specifying Join Columns for Materialized View Logs: Example
The following statement creates a materialized view log on the order_items table of the sample oe schema. The log records primary keys and product_id, which is used as a join column in "Creating a Fast Refreshable Materialized View: Example".
CREATE MATERIALIZED VIEW LOG ON order_items WITH (product_id);

Creating a Fast Refreshable Materialized View: Example
The following statement creates a fast-refreshable materialized view that selects columns from the order_items table in the sample oe schema, using the UNION set operator to restrict the rows returned from the product_information and inventories tables using WHERE conditions. 
The materialized view logs for order_items and product_information were created in the "Examples " section of CREATE MATERIALIZEDVIEW LOG. This example also requires a materialized view log on oe.inventories.

CREATE MATERIALIZED VIEW LOG ON inventories
   WITH (quantity_on_hand);

CREATE MATERIALIZED VIEW warranty_orders REFRESH FAST AS
  SELECT order_id, line_item_id, product_id FROM order_items o
    WHERE EXISTS
    (SELECT * FROM inventories i WHERE o.product_id = i.product_id
      AND i.quantity_on_hand IS NOT NULL)
  UNION
    SELECT order_id, line_item_id, product_id FROM order_items
    WHERE quantity > 5; 


This materialized view requires that materialized view logs be defined on order_items (with product_id as a join column) and on inventories (with quantity_on_hand as a filter column).

Creating Materialized Join Views: Example
The following statement creates and populates the materialized aggregate view sales_by_month_by_state using tables in the sample sh schema. The materialized view will be populated with data as soon as the statement executes successfully. By default, subsequent refreshes will be accomplished by reexecuting the defining query of the materialized view:
CREATE MATERIALIZED VIEW sales_by_month_by_state
     TABLESPACE example
     PARALLEL 4
     BUILD IMMEDIATE
     REFRESH COMPLETE
     ENABLE QUERY REWRITE
     AS SELECT t.calendar_month_desc, c.cust_state_province,
        SUM(s.amount_sold) AS sum_sales
        FROM times t, sales s, customers c
        WHERE s.time_id = t.time_id AND s.cust_id = c.cust_id
        GROUP BY t.calendar_month_desc, c.cust_state_province;

Creating Rowid Materialized Views: Example
The following statement creates a rowid materialized view on the sample table oe.orders:
CREATE MATERIALIZED VIEW order_data REFRESH WITH ROWID 
   AS SELECT * FROM orders;

Periodic Refresh of Materialized Views: Example
The following statement creates the primary key materialized view emp_data and populates it with data from the sample table hr.employees:
CREATE MATERIALIZED VIEW LOG ON employees
   WITH PRIMARY KEY
   INCLUDING NEW VALUES;

CREATE MATERIALIZED VIEW emp_data 
   PCTFREE 5 PCTUSED 60 
   TABLESPACE example 
   STORAGE (INITIAL 50K NEXT 50K)
   REFRESH FAST NEXT sysdate + 7 
   AS SELECT * FROM employees; 

The statement does not include a START WITH parameter, so Oracle Database determines the first automatic refresh time by evaluating the NEXT value using the current SYSDATE. A materialized view log was created for the employee table, so Oracle Database performs a fast refresh of the materialized view every 7 days, beginning 7 days after the materialized view is created.
Because the materialized view conforms to the conditions for fast refresh, the database will perform a fast refresh. The preceding statement also establishes storage characteristics that the database uses to maintain the materialized view.
Automatic Refresh Times for Materialized Views: Example
The following statement creates the complex materialized view all_customers that queries the employee tables on the remote and local databases:
CREATE MATERIALIZED VIEW all_customers
   PCTFREE 5 PCTUSED 60 
   TABLESPACE example 
   STORAGE (INITIAL 50K NEXT 50K) 
   USING INDEX STORAGE (INITIAL 25K NEXT 25K)
   REFRESH START WITH ROUND(SYSDATE + 1) + 11/24 
   NEXT NEXT_DAY(TRUNC(SYSDATE), 'MONDAY') + 15/24 
   AS SELECT * FROM sh.customers@remote 
         UNION
      SELECT * FROM sh.customers@local; 

Oracle Database automatically refreshes this materialized view tomorrow at 11:00 a.m. and subsequently every Monday at 3:00 p.m. The default refresh method is FORCE. The defining query contains a UNION operator, which is not supported for fast refresh, so the database will automatically perform a complete refresh.
The preceding statement also establishes storage characteristics for both the materialized view and the index that the database uses to maintain it:
  • The first STORAGE clause establishes the sizes of the first and second extents of the materialized view as 50 kilobytes each.
  • The second STORAGE clause, appearing with the USING INDEX clause, establishes the sizes of the first and second extents of the index as 25 kilobytes each.

Materialized View REFRESH option

In this post I will be creating read only materialized view and giving different REFRESH option to get data updated in Master table into Materialized view.

Here, user SCOTT will be creating materialized view on HR.EMPLOYEES table. 

Manual Refresh using DBMS_MVIEW, DBMS_SNAPSHOT packages.

First, Grant needful privilege to SCOTT from SYS and HR schema.
---- As SYS ---
GRANT CREATE MATERIALIZED VIEW to SCOTT;

--- As HR ---
GRANT SELECT ON EMPLOYEES to SCOTT; 

-- As SCOTT --
SQL> CREATE MATERIALIZED VIEW scott.hremp
  2  AS SELECT * FROM HR.EMPLOYEES;

Materialized view created.

Querying Materialized view SCOTT.hremp
--- As Scott -- 
SQL> SELECT SALARY FROM HREMP
  2  WHERE EMPLOYEE_ID = 206;
    SALARY
----------
      8500

Now to refresh the materialized view:

Suppose you update one record in Employees Table:
--- As HR -- 
SQL> update employees set salary=8300 where employee_id =206;
1 row updated.

SQL> Commit;

SQL>  SELECT SALARY FROM EMPLOYEES WHERE EMPLOYEE_ID = 206;
    SALARY
----------
      8300

-- As Scott
SQL> SELECT SALARY FROM HREMP WHERE EMPLOYEE_ID = 206;
    SALARY
----------

      8500

Now perform manual refresh using REFRESH method of DBMS_MVIEW package. 

SQL> exec DBMS_MVIEW.REFRESH('HREMP');
PL/SQL procedure successfully completed.

SQL>  SELECT SALARY FROM HREMP WHERE EMPLOYEE_ID = 206;
    SALARY
----------
      8300

Now we got refreshed data from HR.EMPLOYEES table into our materilized view SCOTT.HREMP.

Creating a Materialized View Refresh On commit on Master table:

We will be creating a materialized view again as above. Now in this case whenever commit on changes performed on master table (HR.EMPLOYEES) table is performed, our Materialized view (SCOTT.HREMP) should be refreshed quickly.

This is tricky and need some extra objects and privileges to come into picture.

We should have CREATE MATERIALIZED VIEW privilege, that can be verified as:
-- As SCOTT ---
SQL> SELECT * FROM SESSION_PRIVS;

PRIVILEGE
----------------------------------------
CREATE SESSION
UNLIMITED TABLESPACE
CREATE TABLE
CREATE CLUSTER
CREATE SEQUENCE
CREATE PROCEDURE
CREATE TRIGGER
CREATE MATERIALIZED VIEW
CREATE TYPE
CREATE OPERATOR
CREATE INDEXTYPE 


11 rows selected.

Now from HR schema we also need to create MATERIALIZED VIEW LOG table and GRANT permission on it to SCOTT.

--- AS HR -- 

First drop any already existing MView Log

SQL> DROP  MATERIALIZED VIEW LOG ON HR.EMPLOYEES;
Materialized view log dropped. 

Now create Materialized View Log:

SQL> CREATE MATERIALIZED VIEW LOG ON HR.EMPLOYEES;
Materialized view log created.

This will create a table: MLOG$_. Here MLOG$_EMPLOYEES will be created.

SQL> SELECT * FROM MLOG$_EMPLOYEES;
no rows selected

SQL> DESC MLOG$_EMPLOYEES;
 Name
------------------------------------
 EMPLOYEE_ID
 SNAPTIME$$
 DMLTYPE$$
 OLD_NEW$$
 CHANGE_VECTOR$$
 XID$$

Now we need to GRANT permissions to HR. 

First revoke any already present privilege:

--- AS HR -- 
SQL> REVOKE ALL ON EMPLOYEES FROM SCOTT;
Revoke succeeded.

Now GRANT the needful permissions to SCOTT.

SQL> GRANT SELECT ON EMPLOYEES TO SCOTT;
Grant succeeded.

SQL> GRANT SELECT ON MLOG$_EMPLOYEES TO SCOTT;
Grant succeeded.

Now lets try to create materialized view: 

-- AS SCOTT
SQL> CREATE MATERIALIZED VIEW  scott.hremp
  2          REFRESH FAST ON COMMIT
  3    AS  SELECT * FROM HR.EMPLOYEES;
  AS  SELECT * FROM HR.EMPLOYEES
                       *
ERROR at line 3:
ORA-01031: insufficient privileges

SCOTT need more permission on HR.EMPLOYEES table.

-- As HR
SQL> GRANT ALL ON HR.EMPLOYEES TO SCOTT;

Grant succeeded.

-- As SCOTT, Trying to create M View again

SQL> CREATE MATERIALIZED VIEW  SCOTT.HREMP
  2  REFRESH FAST ON COMMIT
  3  AS  SELECT * FROM HR.EMPLOYEES;
Materialized view created.

Fetching data from M View (SCOTT.HREMP) :
SQL> SELECT SALARY FROM SCOTT.HREMP
  2  WHERE EMPLOYEE_ID = 206;
    SALARY
----------
      8300

Perform changes to some records in employees table followed by commit.

-- As HR
SQL> update employees set salary=salary+200 where employee_id =206;
1 row updated.

SQL> Commit;

SQL> SELECT SALARY FROM HR.EMPLOYEES
  2  WHERE EMPLOYEE_ID = 206;

    SALARY
----------
      8500

-- As SCOTT -- 
SQL> SELECT SALARY FROM SCOTT.HREMP  WHERE EMPLOYEE_ID = 206;
    SALARY
----------
      8500

Immediately data get refreshed.

Now scheduling Materialized View Refresh


Understanding Ref Cursors

Often on the forum we get a question along the lines of:
"I'm passing a ref cursor back from a function, how do I use the data in that ref cursor in my select statement/query"
And, equally as often, the problem comes from a lack of understanding what a ref cursor actually is.  So I thought I'd give a quick summary of the basics behind ref cursors with a view to giving a clearer understanding of them. (Please excuse any errors or omissions)

1. What is a ref cursor and how is the data fetched

Let's start with a simple function that opens a ref cursor and passes it back.
(We'll just select some employee details for a specified department)
SQL> create or replace function get_dept_emps(p_deptno in number) return sys_refcursor is
  2    v_rc sys_refcursor;
  3  begin
  4    open v_rc for 'select empno, ename, mgr, sal from emp where deptno = :deptno' using p_deptno;
  5    return v_rc;
  6  end;
  7  /
Function created.
Now, if we look at using this through SQL*Plus we first create ourselves a ref cursor variable to accept the results of the function, and then call the function to get the ref cursor back..
(Note: SQL*Plus variable type of "refcursor" is the equivalent of PL/SQL's "sys_refcursor")
SQL> var rc refcursor
SQL> exec :rc := get_dept_emps(10);
PL/SQL procedure successfully completed.
Ok, so our variable "rc" has our ref cursor.
If we use SQL*Plus' print command now we get..
SQL> print rc;
     EMPNO ENAME             MGR        SAL
---------- ---------- ---------- ----------
      7782 CLARK            7839       2450
      7839 KING                        5000
      7934 MILLER           7782       1300
Brilliant, so our ref cursor returns the rows we wanted from the employee table.

Let's look at that again..
SQL> print rc;
SP2-0625: Error printing variable "rc".

Uh oh! What's happened here?  Why can't we print the data that's in our ref cursor again?
A common mistake is that people believe that a ref cursor actually contains the result data from the query. In truth, the ref cursor doesn't contain any data at all, it's just a pointer to the query.

So why did the first print statement print out the results?
SQL*Plus looked at the ref cursor and saw that it was an open cursor.  As such it went into a loop, fetching each row of data from the database, using the ref cursor as it's reference (pointer) to the relevant query, and displaying each row of data until it had no more rows to fetch.  Once it's fetched all the rows it closes the cursor.  The power of SQL*Plus's "print" command eh!  

Therefore, when we tried to print the ref cursor a second time, we got an error because SQL*Plus looked at the cursor, saw it was not an open cursor and couldn't perform the task of printing anything. Let's look at this in a bit more detail so it's a little clearer.

We'll use some PL/SQL code so that we can see the type of thing SQL*Plus is doing internally.
Firstly, let's get the open cursor and ask it how many rows it has..
SQL> ed
Wrote file afiedt.buf
  1  declare
  2    v_rc    sys_refcursor;
  3  begin
  4    v_rc := get_dept_emps(10);  -- This returns an open cursor
  5    dbms_output.put_line('Rows: '||v_rc%ROWCOUNT);
  6    close v_rc;
  7* end;
SQL> /
Rows: 0
PL/SQL procedure successfully completed.
Yep, sure enough it's reporting 0 rows.

It's important that we remember that ROWCOUNT reports how many rows have been fetched through the cursor.  Just after opening the cursor we haven't fetched any rows yet.  If we fetch a row of data then we can see this change.
SQL> ed
Wrote file afiedt.buf
  1  declare
  2    v_rc    sys_refcursor;
  3    v_empno number;
  4    v_ename varchar2(10);
  5    v_mgr   number;
  6    v_sal   number;
  7  begin
  8    v_rc := get_dept_emps(10);  -- This returns an open cursor
  9    dbms_output.put_line('Pre Fetch: Rows: '||v_rc%ROWCOUNT);
10    fetch v_rc into v_empno, v_ename, v_mgr, v_sal;
11    dbms_output.put_line('Post Fetch: Rows: '||v_rc%ROWCOUNT);
12    close v_rc;
13* end;
SQL> /
Pre Fetch: Rows: 0
Post Fetch: Rows: 1
PL/SQL procedure successfully completed.

So let's fetch all our data and display it.
SQL> ed
Wrote file afiedt.buf
  1  declare
  2    v_rc    sys_refcursor;
  3    v_empno number;
  4    v_ename varchar2(10);
  5    v_mgr   number;
  6    v_sal   number;
  7  begin
  8    v_rc := get_dept_emps(10);  -- This returns an open cursor
  9    loop
10      fetch v_rc into v_empno, v_ename, v_mgr, v_sal;
11      exit when v_rc%NOTFOUND;  -- Exit the loop when we've run out of data
12      dbms_output.put_line('Row: '||v_rc%ROWCOUNT||' # '||v_empno||','||v_ename||','||v_mgr||','||v_sal);
13    end loop;
14    close v_rc;
15* end;
SQL> /
Row: 1 # 7782,CLARK,7839,2450
Row: 2 # 7839,KING,,5000
Row: 3 # 7934,MILLER,7782,1300
PL/SQL procedure successfully completed.

And what happens if we try and fetch more data after it's finished, just like we tried to do in SQL*Plus..
SQL> ed
Wrote file afiedt.buf
  1  declare
  2    v_rc    sys_refcursor;
  3    v_empno number;
  4    v_ename varchar2(10);
  5    v_mgr   number;
  6    v_sal   number;
  7  begin
  8    v_rc := get_dept_emps(10);  -- This returns an open cursor
  9    loop
10      fetch v_rc into v_empno, v_ename, v_mgr, v_sal;
11      exit when v_rc%NOTFOUND;  -- Exit the loop when we've run out of data
12      dbms_output.put_line('Row: '||v_rc%ROWCOUNT||' # '||v_empno||','||v_ename||','||v_mgr||','||v_sal);
13    end loop;
14    close v_rc;
15    fetch v_rc into v_empno, v_ename, v_mgr, v_sal;
16* end;
SQL> /
Row: 1 # 7782,CLARK,7839,2450
Row: 2 # 7839,KING,,5000
Row: 3 # 7934,MILLER,7782,1300
declare
*
ERROR at line 1:
ORA-01001: invalid cursor
ORA-06512: at line 15
As expected we get an error.  
So now we understand the basics of what a ref cursor is.
You can clearly see that it is just a pointer to the query and it doesn't contain any data itself, it just allows us to reference the query so that we can fetch data as we require it.

2. How can we use a ref cursor in a SQL query?

Ok, so now you've got your ref cursor you want to use it.  But how can you fetch the data from the ref cursor inside another SQL Select statement?
Perhaps we can just select from it?
SQL> select * from get_dept_emps(10);
select * from get_dept_emps(10)
                           *
ERROR at line 1:
ORA-00933: SQL command not properly ended
Nope.  How about if we tell SQL to treat it as a table?
SQL> select * from table(get_dept_emps(10));
select * from table(get_dept_emps(10))
              *
ERROR at line 1:
ORA-22905: cannot access rows from a non-nested table item
What about using it as a set of data in an IN condition?
SQL> select * from emp where empno in (get_dept_emps(10));
select * from emp where empno in (get_dept_emps(10))
                                  *
ERROR at line 1:
ORA-00932: inconsistent datatypes: expected - got CURSER
N.B. The spelling error of "CURSER" is Oracle's, not mine.  

(Ok, I know the ref cursor is referencing more than just the empno in that example, but it would still result in the same error if it just referenced the empno)

The problem we're having is because the ref cursor isn't a table of data either in the literal database sense or in an array sense, and it's not a set of data that can be compared with the IN clause.
So what's the point in these ref cursors? Is there a way we can use them?
Yes there is..
First let's create a type structure on the database.  Remember, SQL cannot access PL/SQL table structures so the type must be a database object..
SQL> create or replace type emptype as object(empno number,
  2                                           ename varchar2(10),
  3                                           mgr   number,
  4                                           sal   number);
  5  /
Type created.
SQL> create or replace type t_emptype as table of emptype;
  2  /
Type created.
Ok, so we have a structure to hold a record and a type that is a table of that structure.  So far so good.  But in order to populate that structure with data coming from the ref cursor we can't just select from it as we saw above.  Instead we need to provide some PL/SQL to actually do the fetching of data for us and populate the structure..
SQL> ed
Wrote file afiedt.buf
  1  create or replace function populate_emps(deptno in number := null)
  2  return t_emptype is
  3    v_emptype t_emptype := t_emptype();  -- Declare a local table structure and initialize it
  4    v_cnt     number := 0;
  5    v_rc      sys_refcursor;
  6    v_empno   number;
  7    v_ename   varchar2(10);
  8    v_mgr     number;
  9    v_sal     number;
10  begin
11    v_rc := get_dept_emps(deptno);
12    loop
13      fetch v_rc into v_empno, v_ename, v_mgr, v_sal;
14      exit when v_rc%NOTFOUND;
15      v_emptype.extend;
16      v_cnt := v_cnt + 1;
17      v_emptype(v_cnt) := emptype(v_empno, v_ename, v_mgr, v_sal);
18    end loop;
19    close v_rc;
20    return v_emptype;
21* end;
SQL> /
Function created.
The above function calls the function that opens the ref cursor, then loops through, fetching each row and populating our SQL type structure.  When all rows have been fetched, the ref cursor is closed and the SQL table structure is passed back from the function.
So now we have something in an structure that SQL understands, we should be able to query directly from it..
SQL> select * from table(populate_emps(30));
     EMPNO ENAME             MGR        SAL
---------- ---------- ---------- ----------
      7499 ALLEN            7698       1600
      7521 WARD             7698       1250
      7654 MARTIN           7698       1250
      7698 BLAKE            7839       2850
      7844 TURNER           7698       1500
      7900 JAMES            7698        950
6 rows selected.
and
SQL> select * from emp where empno in (select empno from table(populate_emps(30)));
     EMPNO ENAME      JOB              MGR HIREDATE           SAL       COMM     DEPTNO
---------- ---------- --------- ---------- ----------- ---------- ---------- ----------
      7499 ALLEN      SALESMAN        7698 20-FEB-1981       1600        300         30
      7521 WARD       SALESMAN        7698 22-FEB-1981       1250        500         30
      7654 MARTIN     SALESMAN        7698 28-SEP-1981       1250       1400         30
      7698 BLAKE      MANAGER         7839 01-MAY-1981       2850                    30
      7844 TURNER     SALESMAN        7698 08-SEP-1981       1500          0         30
      7900 JAMES      CLERK           7698 03-DEC-1981        950                    30
6 rows selected.
SQL>
Hoorah!

We've successfully taken our ref cursor (pointer) and used it to fetch the data back that we want in a structure that SQL can understand.  Ok, the examples are pretty meaningless as they stand as we could easily have achieved the same results through a basic select, but the method is what is important to understand here.

3. What is the point of ref cursors?

A good question.  Many people learn about ref cursors and then try and use them everywhere.  Although I personally know how to write and use ref cursors, I've found very little use for them in my production code. If you are thinking of using a ref cursor, the first thing you should ask yourself is "Why do I need one?"
If your answer is that you need to write your code to dynamically generate SQL, then perhaps you should look at what you are trying to achieve.  Typically dynamic SQL, in a lot of cases, is unnecessary and a major cause of unmaintainable and unscalable code as well as possible performance issues and SQL injection.
If you really have a valid need to dynamically create SQL and you fully understand the implications and risks involved, then a ref cursor is useful for this task.  What you will find however is that you are limited with ref cursors to a fixed result structure, so it may not be as generic a solution as you had initially planned for.  If you consider the examples from above, you will see that we had to define a SQL structure of known columns in order to receive the data from the ref cursor.  So, whilst the function that opens the ref cursor could do so for any piece of query text, the only way of successfully getting the data out of that ref cursor is to know what columns we are expecting back from it.
There is a way around this however.  Either use the DBMS_SQL package to dynamically generate and process your queries or, from 11g onwards, take your ref cursor and convert it to a DBMS_SQL cursor using 11g's new DBMS_SQL.TO_CURSOR_NUMBER feature (http://download.oracle.com/docs/cd/B28359_01/appdev.111/b28419/d_sql.htm#CHDJDGDG).
DBMS_SQL is a very powerful way of writing dynamic SQL, but that's for another article.

You Might Also Like

Related Posts with Thumbnails

Pages