SGPro generated flats problem. (In Pixinsight and CCDStack)

New user here and I love the software!

I am though encountering a very odd problem with the Flats FIT’s files coming out of SGPro. These flats are generated from a luminescent panel on a F7 180mm refractor exposed at around 27,000 ADU. They are pretty textbook and look fine visually.

When I attempt to calibrate these FIT’s images in Pixinsight I keep getting a warning “No flat frames have been selected to calibrate light frames” even though the files are properly selected. Thinking it may be a Pixinsight problem I did a comparison test in CCDStackt, and though the flat FIT’s files can be read they are not subtracting properly from the Light images. (Dust bunnies are still visible)

I do not appear to be having this problem with the bias and dark frames.

I am starting to wonder if there is something in the format of the Flat FIT’s files themselves being exported from SGPro. Either a variant of the FIT’s format that is not matching the Light frames or perhaps something in the file name format itself. Perhaps some output setting?

Has anyone encountered anything like this? I fully accept I could be doing something basically dumb!

Chris

In SGP are you setting the frame type to FLAT? This populates some information in he FITS header that PI is looking for to sort the frame types.

Thanks,
Jared

Hi Chris,
Unfortunately I don’t have a direct answer for you and Ken or Jared will need to weigh in here. I’m pretty sure that one or both of them uses pixinsight though so I’d be surprised if SGP is the issue.

However, lately I have been experiencing a flats problem as well that I assumed is user error but thought I’d mention it. I use DSS for calibration and stacking and lately I’m still getting vignetting issues in the corners after calibration. It has appeared to me that the opposite corners are being corrected than should be. So the result looks like an over correction in one corner and an under correction in the opposite corner. I only mention this as another data point but I still think it’s something I’m doing and not SGP.

Thanks! Yes I do set the frame type to FLAT in SGP which puts the images into the right directory, though no actual FLAT designation appears in the file name when written out. i.e. It comes out as NGC7023_1.3sec_1X1_L_frame1 etc.

I was wondering whether Pixinsight was looking for a FLAT in the file name. However that would not explain why when I calibrate in CCDStack it reads the files correctly, but does not subtract properly. I am wondering if there is actually a file/data mismatch.

Is there any output setting for a flat file that can be set distinctly from light files? i.e.: 32 bit float or integer etc. I was wondering if that is where there is some mismatch leading to problems with calibrating the flat against the light files.

Chris

Interesting. it should still have the type designation in the filename if you have that set for other frames.

Ken and I both use PI, but admittedly I haven’t been processing much lately. I’ll see if I can duplicate it this evening. I doubt it’s the type. I’m pretty sure that if you’re using the PI script to “auto combine” (can’t recall what this is named) then it’s likely looking for the type in the header and not finding it.

Thanks,
Jared

HI,

This is most likely because the light and flat are not in the same hierarchy, are lights taken with a filter?

When you load the lights are they under just ‘Binning 1’ or is the a filter section underneath, say ‘Ha’?

You flats when when loaded have to match hierarchy of your lights.

Easy to sort, use ‘Add Custom’ and chose the type and filter.

Hopefully that is it.

Trevor

Hi

as Trevor said you must make sure that all filter names-binning info is correct as Pixinsight is fussy when trying to match
flats with lights
I have just tried pixinsight and it works fine if you have been a good boy with your naming strategy , but as stated before if
all else fails use the Add custom to get pixinsight to use the flats correctly

Kind regards

Harry

Hi Chris

I use PI and have had no problem using flats if the file name is correct.

What I have done a couple of times which resulted in a misnamed file, is to re-designate an event within a sequence, e.g. change the ‘Type’ from a completed ‘Lum’ event to ‘FLAT’. When I did this the ‘Filter’ didn’t change and nor did the ‘Suffix’ (if you are designating a new event these field labels will change automatically when you designate the ‘Type’ but do not change automatically when you re-designate a complete event). This resulted in an incorrect filename for me and caused some confusion until I figured it out.

Is this something you have done?

Regards,

Barry

When you use batch preprocessing or manual integration in PI with flats it forgets to make the master file a flat with a filter type. You have to manually add it as the type and filter and it’ll work just fine. Check the fits header.

It frustrates me a lot :slight_smile:

Thanks for your help everyone! I am thinking now it must be something to do with the filenames. Stuart another of our SGPro cronies did a test on one each of my light and flat on his machine in both CCDStack and Pixinsight and the flat fields worked perfectly - so I know the data is good. Things appear to be pointing to the file name conventions on my Windows 7 laptop. The frame files I was using from SGPro are in the format below. I DO specify the type in the Sequence Generator, however there is nothing in the filename string itself which says what type the file is, only the directory in which it is deposited.

Example flat: NGC7023_1.3sec_1x1_L_frame1.fit
Example light: NGC7023_600sec_1x1_R_frame1.fit
Example dark: NOV2014_600sec_1x1__frame1.fit
Example bias: NOV2014_0sec_1x1__frame1.fit

I will look again tomorrow, but it appears that filename conventions are the most probable issue here.

Chris

Hi Chris,

Pretty sure file name is irrelevant, PI is looking at the FITS header to retrieve this information.

It is checking IMAGETYP and FILTER keys, open the light and flat in SGP, right click and select Show FITS Headers and check these values.

If you light has values of LIGHT and R, you flat must have values on FLAT and R

Taking light files and assuming values of LIGHT and R in the FITS header for the file NGC7023_600sec_1x1_R_frame1.fit it would load as this

Binning 1
R
NGC7023_600sec_1x1_R_frame1.fit

Taking your files names and assuming values of LIGHT and R in the FITS header for the file NOV2014_600sec_1x1__frame1.fit it would load as this

Binning 1
NGC7023_1.3sec_1x1_L_frame1.fit

PI would object and throw the error you are seeing

The flat frame needs to be loaded as

Binning 1
R
NGC7023_1.3sec_1x1_L_frame1.fit

You can do that by using the Add Custom option in the script, select all your flats, set the type to Flat Field and type R in the filter name, can ignore binning and exposure.

Thanks

Trevor

1 Like

I agree, I also use SGP to make flats and typically use a file renaming program to give the files simpler and easier to understand names before bringing them into PI to calibrate and then combine into a master flat. The renaming does not conform to any special format, just whatever makes sense to me at the time. I would think if filename was critical, this would have caused problems - but it has not.

FYI, I do all of my preprocessing manually (including calibration/combination of flats), not using batch. I just like to know exactly what is being done at each and every step.

1 Like

Success and thanks everyone@ I used the Add Custom Option in the BPP script this morning to specify the exact image and filter type, and everything worked fine. So it was a FIT’s file header problem all along.

A little bit more fiddly from a workflow POV, but the problem is more at the PI end. YOu all saved me some time in hunting this down.

I often calibrate manually also, however the BPP script is very useful for fast previews and checks on data integrity.

Chris

1 Like