I have created scenario in fiddle where I have to remove duplicate records and keep only 1 record example : for spotted table mfg_lz.icc_axbi_kanban_job
there are 2 records so need to remove duplicate and keep only 1.
select field1
,unnest(
regexp_matches(
replace(
replace(
concat(''''
||replace(
replace(
ltrim(rtrim(field2))
,'"'
,''
)
,''''
,''
)
||''''
)
,'"'
,''
)
,E'n'
,'')
--,'(?:UPDATE|INTO|FROM|JOIN|USING|TABLE)(?:s+ONLY)*s+([[:alpha:]._]+[[:word:]]*|"[^"]+")'
,'(?:FROM)(?:s+ONLY)*s+([[:alpha:]._]+[[:word:]]*|"[^"]+")','gi'
)
) AS spotted_table
,field2
from table1;
field1 | spotted_table | field2 |
---|---|---|
a | mfg_lz.icc_axbi_kanban_job | select service_management.analyze_table(kanban_job,mfg_lz_inc);DELETE FROM mfg_lz.icc_axbi_kanban_job WHERE dw_upd_dtsz < To_Date(Cast(extract (year from (Current_Timestamp))-7 AS CHAR(4))||/02/01,yyyy/mm/dd);DELETE FROM mfg_lz.icc_axbi_kanban_job AWHERE EXISTS (SELECT 1 FROM mfg_lz_inc.icc_axbi_kanban_job B WHERE A.recid = B.recid and A.KANBAN=B.KANBAN);INSERT into mfg_lz.icc_axbi_kanban_job(ACTUALENDDATETIME,ACTUALENDDATETIMETZID,ACTUALPREPAREDDATETIME,ACTUALPREPAREDDATETIMETZID,ACTUALSTARTDATETIME,ACTUALSTARTDATETIMETZID,DEFAULTDIMENSION,DUEDATETIME,DUEDATETIMETZID,EXECUTABLE,EXPECTEDDATETIME,EXPECTEDDATETIMETZID,INVENTDIMDATAAREAID,INVENTDIMID,INVENTLOCATIONDATAAREAID,INVENTLOCATIONID,INVENTTRANSDATAAREAID,INVENTTRANSID,ISSUEINVENTDIMDATAAREAID,ISSUEINVENTDIMID,ISSUEINVENTTRANSDATAAREAID,ISSUEINVENTTRANSID,KANBAN,LEANPRODUCTIONFLOWACTIVITY,LEANSCHEDULEGROUPCOLOR,LEANSCHEDULEGROUPNAME,LEANSCHEDULEITEMRATIO,PLANACTIVITYNAME,QUANTITYORDERED,QUANTITYRECEIVED,QUANTITYSCRAPPED,SEQUENCE,SOURCEDOCUMENTHEADER,STATUS,TYPEE,WMSLOCATIONDATAAREAID,WMSLOCATIONID,WORKCELL,LOADPERCENT,PARTITIONN,RECID,RECVERSION,AXLWIPENDDATETIME,AXLWIPENDDATETIMETZID,AXLESTIMATEDSTARTDATETIME,AXLESTIMATEDSTARTDATETIMETZID,AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,AXLKANBANACTIVITYPLANNEDSTARTDATE,AXLKANBANACTIVITYPLANNEDSTARTDATETIME,AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,AXLKANBANACTIVITYPLANNEDENDDATETIME,AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,AXLKANBANACTIVITYSCHEDULINGENDDATETIME,AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,AXLFIS,MANUALCLEAR,MODIFIEDBY,TERMINAL,MODIFIEDDATETIME,MODIFIEDBY,CREATEDDATETIME,CREATEDBY,KFK_INS_DTSZ,SRC_SYS_NM,DW_ROW_HASH_VAL,DW_SRC_SITE_ID,DW_INS_DTSZ,DW_UPD_DTSZ,DW_LD_GRP_VAL,DW_ETL_SESS_NM) select INC_L.ACTUALENDDATETIME,INC_L.ACTUALENDDATETIMETZID,INC_L.ACTUALPREPAREDDATETIME,INC_L.ACTUALPREPAREDDATETIMETZID,INC_L.ACTUALSTARTDATETIME,INC_L.ACTUALSTARTDATETIMETZID,INC_L.DEFAULTDIMENSION,INC_L.DUEDATETIME,INC_L.DUEDATETIMETZID,INC_L.EXECUTABLE,INC_L.EXPECTEDDATETIME,INC_L.EXPECTEDDATETIMETZID,INC_L.INVENTDIMDATAAREAID,INC_L.INVENTDIMID,INC_L.INVENTLOCATIONDATAAREAID,INC_L.INVENTLOCATIONID,INC_L.INVENTTRANSDATAAREAID,INC_L.INVENTTRANSID,INC_L.ISSUEINVENTDIMDATAAREAID,INC_L.ISSUEINVENTDIMID,INC_L.ISSUEINVENTTRANSDATAAREAID,INC_L.ISSUEINVENTTRANSID,INC_L.KANBAN,INC_L.LEANPRODUCTIONFLOWACTIVITY,INC_L.LEANSCHEDULEGROUPCOLOR,INC_L.LEANSCHEDULEGROUPNAME,INC_L.LEANSCHEDULEITEMRATIO,INC_L.PLANACTIVITYNAME,INC_L.QUANTITYORDERED,INC_L.QUANTITYRECEIVED,INC_L.QUANTITYSCRAPPED,INC_L.SEQUENCE,INC_L.SOURCEDOCUMENTHEADER,INC_L.STATUS,INC_L.TYPEE,INC_L.WMSLOCATIONDATAAREAID,INC_L.WMSLOCATIONID,INC_L.WORKCELL,INC_L.LOADPERCENT,INC_L.PARTITIONN,INC_L.RECID,INC_L.RECVERSION,INC_L.AXLWIPENDDATETIME,INC_L.AXLWIPENDDATETIMETZID,INC_L.AXLESTIMATEDSTARTDATETIME,INC_L.AXLESTIMATEDSTARTDATETIMETZID,INC_L.AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATE,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATETIME,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,INC_L.AXLKANBANACTIVITYPLANNEDENDDATETIME,INC_L.AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,INC_L.AXLKANBANACTIVITYSCHEDULINGENDDATETIME,INC_L.AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,INC_L.AXLFIS,INC_L.MANUALCLEAR,INC_L.MODIFIEDBY,INC_L.TERMINAL,INC_L.MODIFIEDDATETIME,INC_L.MODIFIEDBY,INC_L.CREATEDDATETIME,INC_L.CREATEDBY,INC_L.KFK_INS_DTSZ,COALESCE(INC_L.src_sys_nm, ) AS src_sys_nm,md5(coalesce(cast(INC_L.recid as varchar(100))||cast(INC_L.KANBAN as varchar(100))||cast(INC_L.src_sys_nm as varchar(100)),1)) as dw_row_hash_val,1040 as dw_src_site_id,current_timestamp as dw_ins_dtsz,current_timestamp as dw_upd_dtsz,cast((to_char(current_timestamp, YYYYMMDDHH24MI)||.00) as numeric) as dw_ld_grp_val,ICM_INVSSF_DDLDSC_KFK_GPSS_ICC_AXBI_KANBAN_JOB as dw_etl_sess_nm from (select ACTUALENDDATETIME,ACTUALENDDATETIMETZID,ACTUALPREPAREDDATETIME,ACTUALPREPAREDDATETIMETZID,ACTUALSTARTDATETIME,ACTUALSTARTDATETIMETZID,DEFAULTDIMENSION,DUEDATETIME,DUEDATETIMETZID,EXECUTABLE,EXPECTEDDATETIME,EXPECTEDDATETIMETZID,INVENTDIMDATAAREAID,INVENTDIMID,INVENTLOCATIONDATAAREAID,INVENTLOCATIONID,INVENTTRANSDATAAREAID,INVENTTRANSID,ISSUEINVENTDIMDATAAREAID,ISSUEINVENTDIMID,ISSUEINVENTTRANSDATAAREAID,ISSUEINVENTTRANSID,KANBAN,LEANPRODUCTIONFLOWACTIVITY,LEANSCHEDULEGROUPCOLOR,LEANSCHEDULEGROUPNAME,LEANSCHEDULEITEMRATIO,PLANACTIVITYNAME,QUANTITYORDERED,QUANTITYRECEIVED,QUANTITYSCRAPPED,SEQUENCE,SOURCEDOCUMENTHEADER,STATUS,TYPEE,WMSLOCATIONDATAAREAID,WMSLOCATIONID,WORKCELL,LOADPERCENT,PARTITIONN,RECID,RECVERSION,AXLWIPENDDATETIME,AXLWIPENDDATETIMETZID,AXLESTIMATEDSTARTDATETIME,AXLESTIMATEDSTARTDATETIMETZID,AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,AXLKANBANACTIVITYPLANNEDSTARTDATE,AXLKANBANACTIVITYPLANNEDSTARTDATETIME,AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,AXLKANBANACTIVITYPLANNEDENDDATETIME,AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,AXLKANBANACTIVITYSCHEDULINGENDDATETIME,AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,AXLFIS,MANUALCLEAR,MODIFIEDBY,TERMINAL,MODIFIEDDATETIME,MODIFIEDBY,CREATEDDATETIME,CREATEDBY,KFK_INS_DTSZ,SRC_SYS_NM,row_number ()over (partition by recid,kanban,src_sys_nm order by kfk_ins_dtsz desc) as first_value from mfg_lz_inc.icc_axbi_kanban_job) INC_Lwhere first_value=1;select service_management.analyze_table(icc_axbi_kanban_job,mfg_lz); |
a | mfg_lz.icc_axbi_kanban_job | select service_management.analyze_table(kanban_job,mfg_lz_inc);DELETE FROM mfg_lz.icc_axbi_kanban_job WHERE dw_upd_dtsz < To_Date(Cast(extract (year from (Current_Timestamp))-7 AS CHAR(4))||/02/01,yyyy/mm/dd);DELETE FROM mfg_lz.icc_axbi_kanban_job AWHERE EXISTS (SELECT 1 FROM mfg_lz_inc.icc_axbi_kanban_job B WHERE A.recid = B.recid and A.KANBAN=B.KANBAN);INSERT into mfg_lz.icc_axbi_kanban_job(ACTUALENDDATETIME,ACTUALENDDATETIMETZID,ACTUALPREPAREDDATETIME,ACTUALPREPAREDDATETIMETZID,ACTUALSTARTDATETIME,ACTUALSTARTDATETIMETZID,DEFAULTDIMENSION,DUEDATETIME,DUEDATETIMETZID,EXECUTABLE,EXPECTEDDATETIME,EXPECTEDDATETIMETZID,INVENTDIMDATAAREAID,INVENTDIMID,INVENTLOCATIONDATAAREAID,INVENTLOCATIONID,INVENTTRANSDATAAREAID,INVENTTRANSID,ISSUEINVENTDIMDATAAREAID,ISSUEINVENTDIMID,ISSUEINVENTTRANSDATAAREAID,ISSUEINVENTTRANSID,KANBAN,LEANPRODUCTIONFLOWACTIVITY,LEANSCHEDULEGROUPCOLOR,LEANSCHEDULEGROUPNAME,LEANSCHEDULEITEMRATIO,PLANACTIVITYNAME,QUANTITYORDERED,QUANTITYRECEIVED,QUANTITYSCRAPPED,SEQUENCE,SOURCEDOCUMENTHEADER,STATUS,TYPEE,WMSLOCATIONDATAAREAID,WMSLOCATIONID,WORKCELL,LOADPERCENT,PARTITIONN,RECID,RECVERSION,AXLWIPENDDATETIME,AXLWIPENDDATETIMETZID,AXLESTIMATEDSTARTDATETIME,AXLESTIMATEDSTARTDATETIMETZID,AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,AXLKANBANACTIVITYPLANNEDSTARTDATE,AXLKANBANACTIVITYPLANNEDSTARTDATETIME,AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,AXLKANBANACTIVITYPLANNEDENDDATETIME,AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,AXLKANBANACTIVITYSCHEDULINGENDDATETIME,AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,AXLFIS,MANUALCLEAR,MODIFIEDBY,TERMINAL,MODIFIEDDATETIME,MODIFIEDBY,CREATEDDATETIME,CREATEDBY,KFK_INS_DTSZ,SRC_SYS_NM,DW_ROW_HASH_VAL,DW_SRC_SITE_ID,DW_INS_DTSZ,DW_UPD_DTSZ,DW_LD_GRP_VAL,DW_ETL_SESS_NM) select INC_L.ACTUALENDDATETIME,INC_L.ACTUALENDDATETIMETZID,INC_L.ACTUALPREPAREDDATETIME,INC_L.ACTUALPREPAREDDATETIMETZID,INC_L.ACTUALSTARTDATETIME,INC_L.ACTUALSTARTDATETIMETZID,INC_L.DEFAULTDIMENSION,INC_L.DUEDATETIME,INC_L.DUEDATETIMETZID,INC_L.EXECUTABLE,INC_L.EXPECTEDDATETIME,INC_L.EXPECTEDDATETIMETZID,INC_L.INVENTDIMDATAAREAID,INC_L.INVENTDIMID,INC_L.INVENTLOCATIONDATAAREAID,INC_L.INVENTLOCATIONID,INC_L.INVENTTRANSDATAAREAID,INC_L.INVENTTRANSID,INC_L.ISSUEINVENTDIMDATAAREAID,INC_L.ISSUEINVENTDIMID,INC_L.ISSUEINVENTTRANSDATAAREAID,INC_L.ISSUEINVENTTRANSID,INC_L.KANBAN,INC_L.LEANPRODUCTIONFLOWACTIVITY,INC_L.LEANSCHEDULEGROUPCOLOR,INC_L.LEANSCHEDULEGROUPNAME,INC_L.LEANSCHEDULEITEMRATIO,INC_L.PLANACTIVITYNAME,INC_L.QUANTITYORDERED,INC_L.QUANTITYRECEIVED,INC_L.QUANTITYSCRAPPED,INC_L.SEQUENCE,INC_L.SOURCEDOCUMENTHEADER,INC_L.STATUS,INC_L.TYPEE,INC_L.WMSLOCATIONDATAAREAID,INC_L.WMSLOCATIONID,INC_L.WORKCELL,INC_L.LOADPERCENT,INC_L.PARTITIONN,INC_L.RECID,INC_L.RECVERSION,INC_L.AXLWIPENDDATETIME,INC_L.AXLWIPENDDATETIMETZID,INC_L.AXLESTIMATEDSTARTDATETIME,INC_L.AXLESTIMATEDSTARTDATETIMETZID,INC_L.AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATE,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATETIME,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,INC_L.AXLKANBANACTIVITYPLANNEDENDDATETIME,INC_L.AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,INC_L.AXLKANBANACTIVITYSCHEDULINGENDDATETIME,INC_L.AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,INC_L.AXLFIS,INC_L.MANUALCLEAR,INC_L.MODIFIEDBY,INC_L.TERMINAL,INC_L.MODIFIEDDATETIME,INC_L.MODIFIEDBY,INC_L.CREATEDDATETIME,INC_L.CREATEDBY,INC_L.KFK_INS_DTSZ,COALESCE(INC_L.src_sys_nm, ) AS src_sys_nm,md5(coalesce(cast(INC_L.recid as varchar(100))||cast(INC_L.KANBAN as varchar(100))||cast(INC_L.src_sys_nm as varchar(100)),1)) as dw_row_hash_val,1040 as dw_src_site_id,current_timestamp as dw_ins_dtsz,current_timestamp as dw_upd_dtsz,cast((to_char(current_timestamp, YYYYMMDDHH24MI)||.00) as numeric) as dw_ld_grp_val,ICM_INVSSF_DDLDSC_KFK_GPSS_ICC_AXBI_KANBAN_JOB as dw_etl_sess_nm from (select ACTUALENDDATETIME,ACTUALENDDATETIMETZID,ACTUALPREPAREDDATETIME,ACTUALPREPAREDDATETIMETZID,ACTUALSTARTDATETIME,ACTUALSTARTDATETIMETZID,DEFAULTDIMENSION,DUEDATETIME,DUEDATETIMETZID,EXECUTABLE,EXPECTEDDATETIME,EXPECTEDDATETIMETZID,INVENTDIMDATAAREAID,INVENTDIMID,INVENTLOCATIONDATAAREAID,INVENTLOCATIONID,INVENTTRANSDATAAREAID,INVENTTRANSID,ISSUEINVENTDIMDATAAREAID,ISSUEINVENTDIMID,ISSUEINVENTTRANSDATAAREAID,ISSUEINVENTTRANSID,KANBAN,LEANPRODUCTIONFLOWACTIVITY,LEANSCHEDULEGROUPCOLOR,LEANSCHEDULEGROUPNAME,LEANSCHEDULEITEMRATIO,PLANACTIVITYNAME,QUANTITYORDERED,QUANTITYRECEIVED,QUANTITYSCRAPPED,SEQUENCE,SOURCEDOCUMENTHEADER,STATUS,TYPEE,WMSLOCATIONDATAAREAID,WMSLOCATIONID,WORKCELL,LOADPERCENT,PARTITIONN,RECID,RECVERSION,AXLWIPENDDATETIME,AXLWIPENDDATETIMETZID,AXLESTIMATEDSTARTDATETIME,AXLESTIMATEDSTARTDATETIMETZID,AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,AXLKANBANACTIVITYPLANNEDSTARTDATE,AXLKANBANACTIVITYPLANNEDSTARTDATETIME,AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,AXLKANBANACTIVITYPLANNEDENDDATETIME,AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,AXLKANBANACTIVITYSCHEDULINGENDDATETIME,AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,AXLFIS,MANUALCLEAR,MODIFIEDBY,TERMINAL,MODIFIEDDATETIME,MODIFIEDBY,CREATEDDATETIME,CREATEDBY,KFK_INS_DTSZ,SRC_SYS_NM,row_number ()over (partition by recid,kanban,src_sys_nm order by kfk_ins_dtsz desc) as first_value from mfg_lz_inc.icc_axbi_kanban_job) INC_Lwhere first_value=1;select service_management.analyze_table(icc_axbi_kanban_job,mfg_lz); |
a | mfg_lz_inc.icc_axbi_kanban_job | select service_management.analyze_table(kanban_job,mfg_lz_inc);DELETE FROM mfg_lz.icc_axbi_kanban_job WHERE dw_upd_dtsz < To_Date(Cast(extract (year from (Current_Timestamp))-7 AS CHAR(4))||/02/01,yyyy/mm/dd);DELETE FROM mfg_lz.icc_axbi_kanban_job AWHERE EXISTS (SELECT 1 FROM mfg_lz_inc.icc_axbi_kanban_job B WHERE A.recid = B.recid and A.KANBAN=B.KANBAN);INSERT into mfg_lz.icc_axbi_kanban_job(ACTUALENDDATETIME,ACTUALENDDATETIMETZID,ACTUALPREPAREDDATETIME,ACTUALPREPAREDDATETIMETZID,ACTUALSTARTDATETIME,ACTUALSTARTDATETIMETZID,DEFAULTDIMENSION,DUEDATETIME,DUEDATETIMETZID,EXECUTABLE,EXPECTEDDATETIME,EXPECTEDDATETIMETZID,INVENTDIMDATAAREAID,INVENTDIMID,INVENTLOCATIONDATAAREAID,INVENTLOCATIONID,INVENTTRANSDATAAREAID,INVENTTRANSID,ISSUEINVENTDIMDATAAREAID,ISSUEINVENTDIMID,ISSUEINVENTTRANSDATAAREAID,ISSUEINVENTTRANSID,KANBAN,LEANPRODUCTIONFLOWACTIVITY,LEANSCHEDULEGROUPCOLOR,LEANSCHEDULEGROUPNAME,LEANSCHEDULEITEMRATIO,PLANACTIVITYNAME,QUANTITYORDERED,QUANTITYRECEIVED,QUANTITYSCRAPPED,SEQUENCE,SOURCEDOCUMENTHEADER,STATUS,TYPEE,WMSLOCATIONDATAAREAID,WMSLOCATIONID,WORKCELL,LOADPERCENT,PARTITIONN,RECID,RECVERSION,AXLWIPENDDATETIME,AXLWIPENDDATETIMETZID,AXLESTIMATEDSTARTDATETIME,AXLESTIMATEDSTARTDATETIMETZID,AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,AXLKANBANACTIVITYPLANNEDSTARTDATE,AXLKANBANACTIVITYPLANNEDSTARTDATETIME,AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,AXLKANBANACTIVITYPLANNEDENDDATETIME,AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,AXLKANBANACTIVITYSCHEDULINGENDDATETIME,AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,AXLFIS,MANUALCLEAR,MODIFIEDBY,TERMINAL,MODIFIEDDATETIME,MODIFIEDBY,CREATEDDATETIME,CREATEDBY,KFK_INS_DTSZ,SRC_SYS_NM,DW_ROW_HASH_VAL,DW_SRC_SITE_ID,DW_INS_DTSZ,DW_UPD_DTSZ,DW_LD_GRP_VAL,DW_ETL_SESS_NM) select INC_L.ACTUALENDDATETIME,INC_L.ACTUALENDDATETIMETZID,INC_L.ACTUALPREPAREDDATETIME,INC_L.ACTUALPREPAREDDATETIMETZID,INC_L.ACTUALSTARTDATETIME,INC_L.ACTUALSTARTDATETIMETZID,INC_L.DEFAULTDIMENSION,INC_L.DUEDATETIME,INC_L.DUEDATETIMETZID,INC_L.EXECUTABLE,INC_L.EXPECTEDDATETIME,INC_L.EXPECTEDDATETIMETZID,INC_L.INVENTDIMDATAAREAID,INC_L.INVENTDIMID,INC_L.INVENTLOCATIONDATAAREAID,INC_L.INVENTLOCATIONID,INC_L.INVENTTRANSDATAAREAID,INC_L.INVENTTRANSID,INC_L.ISSUEINVENTDIMDATAAREAID,INC_L.ISSUEINVENTDIMID,INC_L.ISSUEINVENTTRANSDATAAREAID,INC_L.ISSUEINVENTTRANSID,INC_L.KANBAN,INC_L.LEANPRODUCTIONFLOWACTIVITY,INC_L.LEANSCHEDULEGROUPCOLOR,INC_L.LEANSCHEDULEGROUPNAME,INC_L.LEANSCHEDULEITEMRATIO,INC_L.PLANACTIVITYNAME,INC_L.QUANTITYORDERED,INC_L.QUANTITYRECEIVED,INC_L.QUANTITYSCRAPPED,INC_L.SEQUENCE,INC_L.SOURCEDOCUMENTHEADER,INC_L.STATUS,INC_L.TYPEE,INC_L.WMSLOCATIONDATAAREAID,INC_L.WMSLOCATIONID,INC_L.WORKCELL,INC_L.LOADPERCENT,INC_L.PARTITIONN,INC_L.RECID,INC_L.RECVERSION,INC_L.AXLWIPENDDATETIME,INC_L.AXLWIPENDDATETIMETZID,INC_L.AXLESTIMATEDSTARTDATETIME,INC_L.AXLESTIMATEDSTARTDATETIMETZID,INC_L.AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATE,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATETIME,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,INC_L.AXLKANBANACTIVITYPLANNEDENDDATETIME,INC_L.AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,INC_L.AXLKANBANACTIVITYSCHEDULINGENDDATETIME,INC_L.AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,INC_L.AXLFIS,INC_L.MANUALCLEAR,INC_L.MODIFIEDBY,INC_L.TERMINAL,INC_L.MODIFIEDDATETIME,INC_L.MODIFIEDBY,INC_L.CREATEDDATETIME,INC_L.CREATEDBY,INC_L.KFK_INS_DTSZ,COALESCE(INC_L.src_sys_nm, ) AS src_sys_nm,md5(coalesce(cast(INC_L.recid as varchar(100))||cast(INC_L.KANBAN as varchar(100))||cast(INC_L.src_sys_nm as varchar(100)),1)) as dw_row_hash_val,1040 as dw_src_site_id,current_timestamp as dw_ins_dtsz,current_timestamp as dw_upd_dtsz,cast((to_char(current_timestamp, YYYYMMDDHH24MI)||.00) as numeric) as dw_ld_grp_val,ICM_INVSSF_DDLDSC_KFK_GPSS_ICC_AXBI_KANBAN_JOB as dw_etl_sess_nm from (select ACTUALENDDATETIME,ACTUALENDDATETIMETZID,ACTUALPREPAREDDATETIME,ACTUALPREPAREDDATETIMETZID,ACTUALSTARTDATETIME,ACTUALSTARTDATETIMETZID,DEFAULTDIMENSION,DUEDATETIME,DUEDATETIMETZID,EXECUTABLE,EXPECTEDDATETIME,EXPECTEDDATETIMETZID,INVENTDIMDATAAREAID,INVENTDIMID,INVENTLOCATIONDATAAREAID,INVENTLOCATIONID,INVENTTRANSDATAAREAID,INVENTTRANSID,ISSUEINVENTDIMDATAAREAID,ISSUEINVENTDIMID,ISSUEINVENTTRANSDATAAREAID,ISSUEINVENTTRANSID,KANBAN,LEANPRODUCTIONFLOWACTIVITY,LEANSCHEDULEGROUPCOLOR,LEANSCHEDULEGROUPNAME,LEANSCHEDULEITEMRATIO,PLANACTIVITYNAME,QUANTITYORDERED,QUANTITYRECEIVED,QUANTITYSCRAPPED,SEQUENCE,SOURCEDOCUMENTHEADER,STATUS,TYPEE,WMSLOCATIONDATAAREAID,WMSLOCATIONID,WORKCELL,LOADPERCENT,PARTITIONN,RECID,RECVERSION,AXLWIPENDDATETIME,AXLWIPENDDATETIMETZID,AXLESTIMATEDSTARTDATETIME,AXLESTIMATEDSTARTDATETIMETZID,AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,AXLKANBANACTIVITYPLANNEDSTARTDATE,AXLKANBANACTIVITYPLANNEDSTARTDATETIME,AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,AXLKANBANACTIVITYPLANNEDENDDATETIME,AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,AXLKANBANACTIVITYSCHEDULINGENDDATETIME,AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,AXLFIS,MANUALCLEAR,MODIFIEDBY,TERMINAL,MODIFIEDDATETIME,MODIFIEDBY,CREATEDDATETIME,CREATEDBY,KFK_INS_DTSZ,SRC_SYS_NM,row_number ()over (partition by recid,kanban,src_sys_nm order by kfk_ins_dtsz desc) as first_value from mfg_lz_inc.icc_axbi_kanban_job) INC_Lwhere first_value=1;select service_management.analyze_table(icc_axbi_kanban_job,mfg_lz); |
a | mfg_lz_inc.icc_axbi_kanban_job | select service_management.analyze_table(kanban_job,mfg_lz_inc);DELETE FROM mfg_lz.icc_axbi_kanban_job WHERE dw_upd_dtsz < To_Date(Cast(extract (year from (Current_Timestamp))-7 AS CHAR(4))||/02/01,yyyy/mm/dd);DELETE FROM mfg_lz.icc_axbi_kanban_job AWHERE EXISTS (SELECT 1 FROM mfg_lz_inc.icc_axbi_kanban_job B WHERE A.recid = B.recid and A.KANBAN=B.KANBAN);INSERT into mfg_lz.icc_axbi_kanban_job(ACTUALENDDATETIME,ACTUALENDDATETIMETZID,ACTUALPREPAREDDATETIME,ACTUALPREPAREDDATETIMETZID,ACTUALSTARTDATETIME,ACTUALSTARTDATETIMETZID,DEFAULTDIMENSION,DUEDATETIME,DUEDATETIMETZID,EXECUTABLE,EXPECTEDDATETIME,EXPECTEDDATETIMETZID,INVENTDIMDATAAREAID,INVENTDIMID,INVENTLOCATIONDATAAREAID,INVENTLOCATIONID,INVENTTRANSDATAAREAID,INVENTTRANSID,ISSUEINVENTDIMDATAAREAID,ISSUEINVENTDIMID,ISSUEINVENTTRANSDATAAREAID,ISSUEINVENTTRANSID,KANBAN,LEANPRODUCTIONFLOWACTIVITY,LEANSCHEDULEGROUPCOLOR,LEANSCHEDULEGROUPNAME,LEANSCHEDULEITEMRATIO,PLANACTIVITYNAME,QUANTITYORDERED,QUANTITYRECEIVED,QUANTITYSCRAPPED,SEQUENCE,SOURCEDOCUMENTHEADER,STATUS,TYPEE,WMSLOCATIONDATAAREAID,WMSLOCATIONID,WORKCELL,LOADPERCENT,PARTITIONN,RECID,RECVERSION,AXLWIPENDDATETIME,AXLWIPENDDATETIMETZID,AXLESTIMATEDSTARTDATETIME,AXLESTIMATEDSTARTDATETIMETZID,AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,AXLKANBANACTIVITYPLANNEDSTARTDATE,AXLKANBANACTIVITYPLANNEDSTARTDATETIME,AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,AXLKANBANACTIVITYPLANNEDENDDATETIME,AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,AXLKANBANACTIVITYSCHEDULINGENDDATETIME,AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,AXLFIS,MANUALCLEAR,MODIFIEDBY,TERMINAL,MODIFIEDDATETIME,MODIFIEDBY,CREATEDDATETIME,CREATEDBY,KFK_INS_DTSZ,SRC_SYS_NM,DW_ROW_HASH_VAL,DW_SRC_SITE_ID,DW_INS_DTSZ,DW_UPD_DTSZ,DW_LD_GRP_VAL,DW_ETL_SESS_NM) select INC_L.ACTUALENDDATETIME,INC_L.ACTUALENDDATETIMETZID,INC_L.ACTUALPREPAREDDATETIME,INC_L.ACTUALPREPAREDDATETIMETZID,INC_L.ACTUALSTARTDATETIME,INC_L.ACTUALSTARTDATETIMETZID,INC_L.DEFAULTDIMENSION,INC_L.DUEDATETIME,INC_L.DUEDATETIMETZID,INC_L.EXECUTABLE,INC_L.EXPECTEDDATETIME,INC_L.EXPECTEDDATETIMETZID,INC_L.INVENTDIMDATAAREAID,INC_L.INVENTDIMID,INC_L.INVENTLOCATIONDATAAREAID,INC_L.INVENTLOCATIONID,INC_L.INVENTTRANSDATAAREAID,INC_L.INVENTTRANSID,INC_L.ISSUEINVENTDIMDATAAREAID,INC_L.ISSUEINVENTDIMID,INC_L.ISSUEINVENTTRANSDATAAREAID,INC_L.ISSUEINVENTTRANSID,INC_L.KANBAN,INC_L.LEANPRODUCTIONFLOWACTIVITY,INC_L.LEANSCHEDULEGROUPCOLOR,INC_L.LEANSCHEDULEGROUPNAME,INC_L.LEANSCHEDULEITEMRATIO,INC_L.PLANACTIVITYNAME,INC_L.QUANTITYORDERED,INC_L.QUANTITYRECEIVED,INC_L.QUANTITYSCRAPPED,INC_L.SEQUENCE,INC_L.SOURCEDOCUMENTHEADER,INC_L.STATUS,INC_L.TYPEE,INC_L.WMSLOCATIONDATAAREAID,INC_L.WMSLOCATIONID,INC_L.WORKCELL,INC_L.LOADPERCENT,INC_L.PARTITIONN,INC_L.RECID,INC_L.RECVERSION,INC_L.AXLWIPENDDATETIME,INC_L.AXLWIPENDDATETIMETZID,INC_L.AXLESTIMATEDSTARTDATETIME,INC_L.AXLESTIMATEDSTARTDATETIMETZID,INC_L.AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATE,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATETIME,INC_L.AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,INC_L.AXLKANBANACTIVITYPLANNEDENDDATETIME,INC_L.AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,INC_L.AXLKANBANACTIVITYSCHEDULINGENDDATETIME,INC_L.AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,INC_L.AXLFIS,INC_L.MANUALCLEAR,INC_L.MODIFIEDBY,INC_L.TERMINAL,INC_L.MODIFIEDDATETIME,INC_L.MODIFIEDBY,INC_L.CREATEDDATETIME,INC_L.CREATEDBY,INC_L.KFK_INS_DTSZ,COALESCE(INC_L.src_sys_nm, ) AS src_sys_nm,md5(coalesce(cast(INC_L.recid as varchar(100))||cast(INC_L.KANBAN as varchar(100))||cast(INC_L.src_sys_nm as varchar(100)),1)) as dw_row_hash_val,1040 as dw_src_site_id,current_timestamp as dw_ins_dtsz,current_timestamp as dw_upd_dtsz,cast((to_char(current_timestamp, YYYYMMDDHH24MI)||.00) as numeric) as dw_ld_grp_val,ICM_INVSSF_DDLDSC_KFK_GPSS_ICC_AXBI_KANBAN_JOB as dw_etl_sess_nm from (select ACTUALENDDATETIME,ACTUALENDDATETIMETZID,ACTUALPREPAREDDATETIME,ACTUALPREPAREDDATETIMETZID,ACTUALSTARTDATETIME,ACTUALSTARTDATETIMETZID,DEFAULTDIMENSION,DUEDATETIME,DUEDATETIMETZID,EXECUTABLE,EXPECTEDDATETIME,EXPECTEDDATETIMETZID,INVENTDIMDATAAREAID,INVENTDIMID,INVENTLOCATIONDATAAREAID,INVENTLOCATIONID,INVENTTRANSDATAAREAID,INVENTTRANSID,ISSUEINVENTDIMDATAAREAID,ISSUEINVENTDIMID,ISSUEINVENTTRANSDATAAREAID,ISSUEINVENTTRANSID,KANBAN,LEANPRODUCTIONFLOWACTIVITY,LEANSCHEDULEGROUPCOLOR,LEANSCHEDULEGROUPNAME,LEANSCHEDULEITEMRATIO,PLANACTIVITYNAME,QUANTITYORDERED,QUANTITYRECEIVED,QUANTITYSCRAPPED,SEQUENCE,SOURCEDOCUMENTHEADER,STATUS,TYPEE,WMSLOCATIONDATAAREAID,WMSLOCATIONID,WORKCELL,LOADPERCENT,PARTITIONN,RECID,RECVERSION,AXLWIPENDDATETIME,AXLWIPENDDATETIMETZID,AXLESTIMATEDSTARTDATETIME,AXLESTIMATEDSTARTDATETIMETZID,AXLRESOURCECONSTRAINEDJOBTIMEMINUTES,AXLKANBANACTIVITYPLANNEDSTARTDATE,AXLKANBANACTIVITYPLANNEDSTARTDATETIME,AXLKANBANACTIVITYPLANNEDSTARTDATETIMETZID,AXLKANBANACTIVITYPLANNEDENDDATETIME,AXLKANBANACTIVITYPLANNEDENDDATETIMETZID,AXLKANBANACTIVITYSCHEDULINGENDDATETIME,AXLKANBANACTIVITYSCHEDULINGENDDATETIMETZID,AXLFIS,MANUALCLEAR,MODIFIEDBY,TERMINAL,MODIFIEDDATETIME,MODIFIEDBY,CREATEDDATETIME,CREATEDBY,KFK_INS_DTSZ,SRC_SYS_NM,row_number ()over (partition by recid,kanban,src_sys_nm order by kfk_ins_dtsz desc) as first_value from mfg_lz_inc.icc_axbi_kanban_job) INC_Lwhere first_value=1;select service_management.analyze_table(icc_axbi_kanban_job,mfg_lz); |
Also I don’t need records where spotted table is having inc
schema suffix like mfg_lz_
inc
.icc_axbi_kanban_job
.
So ideally I am expecting only 1 row in output with spotted table mfg_lz.icc_axbi_kanban_job
.
However currently I am getting 4 rows. What should I write in regex to achieve that output?
2
Answers
btrim()
if you want to use bothrtrim()
andltrim()
.replace()
, you can use a singletranslate()
.concat()
with a single argument, that you constructed by concatenating with||
. You only need one of these: separate arguments passed toconcat()
with commas, or removeconcat()
entirely and concatenate with only||
instead.[:word:]
and[:alpha:]
character classes are available in PostgreSQL 14 onwards. Assuming you’re on 9.5 (highest version tag you used), you can swap those out for aw
, downgrading your regex a bit (it now allows identifiers to start with a digit, which is not allowed).distinct on(spotted_table)...order by spotted_table
to get just one row for each distinct table.Demo at db<>fiddle:
@Ram
You can try to remove the duplicates by identifying which duplicates you have.
For that you may do the removal in two phases:
Phase 1: Identification of duplicates:
a)
GROUP BY
clause, to identify only the duplicates;b) and make sure your table’s primary key is correctly set;
c)
ctid
hidden column on PostgreSQL do delete only the one you want;For example:
(presumabily the
primary_key_you_have
is your primary key, so please replace with the one your table have).Phase 2: Removal of duplicates.
In that scenario you will have then to DELETE manually based on your
ctid
.Because of that make sure you take notes beforehand with the output given from
the Phase 1.
For each primary key(s) you use:
Iteract for who many CTID you have in the duplicated output until there is only one, and make sure you don’t deleat the last one. Use the values double-check if they are indeed duplicated.
IMPORTANT: if you don’t have primary keys (and therefore your model doesn’t planned it – so it’s more like a database design problem rather than a database failure and you may have to try a different approach)
An alternative approach would be to add all columns in the group by clause.